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1 Introduction 

Advanced Communication for Wireless Sensor Networks was proposed to focus on the 
development of innovative technologies for networked sensing, information processing, 
and control. This focus was in support of the Army Research Laboratory’s (ARL) research 
and development program in advanced communications for mobile networks. More 
specifically the primary objective was to develop and demonstrate key sensor-network 
capabilities, lower-power electronic technology, and flexible, highly reliable 
communication systems to support small unit military operations. To accomplish this 
objective, the project was divided in to tasks, which included: 

Task 1: Design, test and demonstrate cooperative and multi-antenna communications 
systems. 

Task 2: Design, test, and demonstrate low-power analog and compressive sensing 
subsystems. 

Task 3: Design, test and demonstrate real-time fault-tolerant protocols for data 
aggregation, inference, location tracking, and connectivity maintenance in a mobile 
heterogeneous sensor network. 

Task 4: Design and demonstrate massively parralell simulation of elarge-escale sensor 
networks. 

The accomplishments for each of these tasks are documented in the subsequent sections. 
Furthermore, each task has been subdivided in to subtasks and the activities performed to 
satisfy these subtasks are also presented. 



2 Task #1: Design, test, and demonstrate cooperative and multi¬ 
antenna communications systems. 

2.1 Subtask #1.1 Rapid Prototyping of MIMO Systems 

2.1.1 Initial Steps and Setup 

After acquiring and assembling the USRP-based software radio testbed, the team began 
developing a series of successively more advanced demonstrations. The most basic 
demonstration was FM reception. The next demonstration was voice transmission and 
reception (using analog single-sideband modulation). Next, basic digital transmission and 
reception of text and video was demonstrated (using GMSK modulation). Finally, a GMSK 
transceiver was implemented, which was capable of both transmitting and receiving in the 
same radio. The system was implemented using the Gnu Radio Companion (GRC), which was 
a graphical layout tool that enabled rapid prototyping. A screenshot of the GMSK transceiver, 
as represented in GRC, is shown in Figure 1. 
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Figure 1. GRC representation of the GMSK transceiver. 

The USRP hardware was represented in GRC by either a USRP Source (the received chain) or 
a USRP Sink (the transmitter chain). Figure 2 shows the received signal spectrum and 
decoded video signal. 





































Transceiver 


COO parkrur»1280 23mbps.ts 

Movie Edit View Go Sound Help 


Received Spectrum 



Frequency (GHz) 


Trace Options 

Peak Hold 
G Average 


Persistence 


ni Trace A store 
Trace B store 
Axis Options 
dB/Div: + - 

Ref Level: + - 

Autoscale 
Stop 



Figure 2. Spectrum of received GMSK signal and the corresponding decoded video signal. 

After procuring appropriate software-defined-radio (SDR) hardware, data was successfully 
transmitted using one transmit antenna and two receive antennas. For the test application, 
a real-time audio/video chat function was developed (similar to Skype) for the SDR platform. 

2.1.2 Installation of Simulink Drivers 

Simulink drivers were installed and QPSK modulation was tested using Simulink/USRP. 
With the help of Mathworks, synchronization and frequency/phase compensation issues 
were resolved for a 1 x 1-antenna configuration and audio/video data were transmitted 
successfully. 

Video and image processing with Simulink/USRP platform was performed. Two options 
seemed to be the most feasible during this phase, these options were: 1) adaptive 
image/channel coding based on image segment priorities and 2) the embedding of sensor 
signal processing results as watermarks in images/video. Communications using the 1X2- 
antenna configuration and 1 X 2-antenna configurations were optimized for ARL visitors 
using the GNU/USRP platform. 

Reallocation of some of the SDR testbed efforts from the GNU software platform to 
Matlab/Simulink was perfomred to take advantage of professional support and built-in 
functionality. Although multi-antenna communication was not supported by Mathworks, the 
team able achieve communications with one transmit antenna and two receive antennas 
after resolving some spatial synchronization issues. 

2.1.3 Signal Processing Strategy for Increased Efficiency 

A a general signal processing strategy was developed for increasing energy efficiency in 
surveillance sensor networks. This included; 1) image/video segmentation, where motion 
was extracted from a video stream at the sensor node and, 2) priority encoding of the 
segmented video, where high priority parts of the video (containing motion) were source 
coded for high quality and channel coded for high reliability, and lower priority video 
segments (backgrounds) were allocated fewer bits for source and channel coding. 













Further, an application was created that demonstrated convolutionally-coded 
communications between two USRP radios. The demo app allowed the users to select the 
length of the synchronization preamble and the length of the message, and displayed the 
resulting error rate. 

2.1.4 Dataset 

The ChokePoint Dataset developed by NICTA was selected for the experiments, as images in 
it were very similar to real surveillance videos even though the dataset was collected in a 
controlled environment [1]. 



Figure 3. Camera Rig [1 ]. 


Figure 3 depicts the camera rig utilized for Chokepoint Dataset experiments. The rig was 
stabilized to an aluminum tripod for extra support. 



Figure 4 depicts an individual being photographed by the camera Chokepoint camera rig. As 
this individual rounded the corner and entered the doorframe, his image was captured. 








The famous Open CV Viola-Jones Face Detection algorithm was utilized in Matlab for the 
purpose of performance evaluation [2]. The code was downloaded from the Matlab 
exchange. 

2.1.5 Video Compression 

The most popular and advanced video compression method H.264 encoding (provided by 
MATLAB [3]] was used to compress the video data. Different settings were tested for the 
most balanced point between power consumption and performance. 

2.1.6 Initial Proposed Algorithm 

Historically, few applications need a "live" video feed. Low latency, however, always will be 
the key requirement for smart video sensor because of time sensitivity and the small buffer 
size. Not every frame is important, and not even everything in a single frame is important. 
When the team tried to remove the redundancy in the video data, the team actually defined 
what was important (or useful). Criteria were often different for different applications. 

In the initially proposed algorithm, a face detector was used to detect the face and then a 
quality evaluation box was used to find the best “face" candidate for a certain person. Data 
rate was recorded and analyzed when the proposed algorithm was compared with pure 
video compression based algorithms. Figure 5 depicts the chain of command for this 
algorithm. 


Start 
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Figure 5. The chain of command for the initially proposed algorithm. 














2.1.7 Preliminary Results 



Figure 6. Data rate for the first sample video using the proposed method (in total 246KB]. 



Figure 7. Data rate for the first sample video using H.264 video compression (Quality=l,4.0MB). 































































































































































































Figure 8. Data rate for the first sample video using H.264 video compression (Quality=25,11.9M]. 

Figure 9 is a sample BER plot comparing the performance of the [1,1] and [1,2] antenna 
configurations using the SDR testbed. The array gain that was expected using two receive 
antennas is clearly visible, though there is a BER increase at large gain that was not 
expected. 


BER vs Gain - 1x2 Case 



Figure 9. BER plot comparing the performance of the (1,1] and (1,2) antenna configurations. 








































































Testing was completed of SIMO communications (with one transmit antenna and two 
receive antennas) using the software-defined-radio (SDR) testbed, and a conference paper 
was submitted on this topic to the IEEE Consumer Communications and Networking 
Conference (CCNC 2013). 

Results were produced using MISO communications (using two transmit antennas and one 
receive antenna) with the Alamouti space-time block code. 

2.2 Subtask #1.2 Cooperative Communication Systems 

2.2.1 Image/Video Processing in Software Radio for Increased Range and/or 
Improved Energy Efficiency: 

With respect to image/video processing for this project, the goal was to develop signal 
processing strategies that could be used to increase the single-hop transmission range of a 
wireless sensor network, increase energy efficiency (improve battery life), or both (each to 
a lesser extent). Ultimately, two such strategies were developed along with a roadmap for 
incorporating additional technologies. 

For a video stream, there were several methods available to reduce the size of the final data 
package that would be delivered to the receiver, which, using appropriate power control, 
allowed the range of the network to be extended or provided an increase in its battery life. 
Based on the complexity level (from low to high), some of them were: 

1. Lossless data compression, e.g., "zip” coding 

2. Lossy data compression, e.g., MPEG, MJPEG 

3. Sleep after a certain length of "quite" time and be waked up by other sensors when 
something happens 

4. Frame selection and digital zoom based on the simple motion detection 

5. Frame selection and digital zoom based on the complex event detection 

6. High level feature extraction, such as coordinates within a video frame 

While data processing also consumed some power, it was expected that this would be more 
than offset by improved energy efficiency. The team chose to implement some of these 
strategies using our GNU SDR testbed. In order to do so, several existing code blocks were 
modified and re-designed for the project: 

1. Image Source: In this block, a color or a gray scale image was read into memory and 
passed to the next block, pixel by pixel. The original code could not handle large color 
images. This bug was fixed by carefully checking the buffer size. 

2. Frame Selection: See Figure 10. This is a new signal-processing block 
created by the team. Similar to the image source block, instead of an 
image filename, a video device index was specified in this video 
source block. To save the power, only when there is clear 
motion detected in the scene will the sensor capture and 


Figure 10. Frame 
selection. 


transmit the video at full resolution and rate. Otherwise, the sensor operated at a 
very low frame rate, e.g., 0.2 FPS (frames per second]. This was called video frame 
selection. 

3. Digital Zoom: This was another new signal processing block created by the team. In 
many cases of interest, only a small portion of a video frame contained useful 
information. If nothing occurred in the scene, this block sends a down-sampled frame 
(1/4 of the original size]. If there was clear motion in the scene, this block locates the 
motion and crops the image around the motion location, i.e., a digital zoom function. 
Figure 11 illustrates digital zoom. 



Figure 11. Tracking motion with digital zoom functionality. 


As alluded to earlier, a general signal processing strategy was developed for increasing 
energy efficiency in surveillance sensor networks. This included; 1] image/video 
segmentation, where motion is extracted from a video stream at the sensor node, and 2] 
priority encoding of the segmented video, where high priority parts of the video (containing 
motion] were source coded for high quality and channel coded for high reliability, and lower 
priority video segments (backgrounds] were allocated fewer bits for source and channel 
coding. 









3 Task #2: Design, test, and demonstrate low-power analog and 
compressive sensing subsystems. 

3.1 Subtask 2.1: Nano-Derived Microsensors for Remote Threat Detection 

3.1.1 Nano-Derived Micro-Magnetometer 

The fabrication of the first magnetometer was successful. This magnetometer was made of a 
PZT-spinel composite. PZT plates were successfully fabricated by a tape-casting process 
using commercially available PZT. These plates showed good bulk density (low internal 
porosity) and microstructure (see Figure 12a). In the case of the spinel plates the precursor 
powders were not commercially available; therefore, a coprecipitation synthesis was 
developed to fabricate powder with the proper composition, particle morphology and crystal 
structure. A calcination study was completed in order to obtain a powder with good surface 
area and crystallinity. The x-ray diffraction spectra of the material produced is displayed in 
Figure 12b. With the powders, preparation of the spinel plates was achieved. Different 
thermal processing treatments were investigated in order to obtain a high bulk density of 
the plates. The morphology of the plates and the bulk density for different calcinations 
temperatures can be seen in Figure 12b. Finally, a macroscopic view of both plates can be 
seen in Figure 12c. 

3.1.2 Fabrication of Sensors 

To fabricate the different micro scale sensors proposed, the desired ceramic materials 
needed to be fabricated by micro-scale techniques. In the case of the magnetoelectric 
laminate, the most appropriate technique was through alternate spin coatings of sol gel 
solutions of the two compositions. For the 0-3 composite, a particulate dispersion of 
magnetostrictive nanomaterials within the piezoelectric sol gel solution was required to 
produce the hybrid system. In the case of the PZT composition, a sol gel was successfully 
synthesized. This solution was spin coated showing good viscosity and homogeneity. The 
obtained thin films were characterized by XRD and the results showed the characteristic 
peaks of the PZT crystallography and the presence of non-desired secondary phases. 
Regarding the spinel thin film, a new sol gel process was developed. The obtained solutions 
presented non-desired particulates precipitation. To solve this problem, different precursors 
and solvents were purchased in order to suppress this precipitation. Regarding the spinel 
nano-particles, they were successfully fabricated by a hydrothermal process. The 
morphology of the obtained particles and the XRD characterization can be seen in Figure 
12d. 




Figure 12. Magnetometer accomplishments; a) SEM picture of a PZT plate, b] SEM picture of a spinel plate with his XRD 
and bulk density analysis, c] Macroscopic picture of booth plates and d) SEM and XRD analysis of spinel nano-particles. 

3.1.3 Characterization of Magnetostrictive Plates 

First, the magnetostrictive plates were characterized using a vibrating sample 
magnetometer to produce a magnetostriction in the sample and the deformation was 
measured with a strain gage. A representative curve of the parallel magnetostriction can be 
seen in Figure 13(a). The characterized material shows a comparable behavior as the one 
reported by Sheikh et al [4]. Here, an increase in the magnetostriction with magnetic fields 
up to 0.4 kOe can be seen. This represents the saturation magnetic field of the material. For 
higher magnetic fields, the magnetostriction remained constant. In the case of the PZT, the 
piezoelectric properties were measured using a high voltage power supply to pole the 
sample and a linear variable differential transformer to measure the deformation. These 
measurements were made in the installations of TU Darmstadt, Germany. The obtained 
results are shown in Figure 13(b). A linear response of the piezoelectric response with the 
applied electric field is shown. The obtained results are comparable to the ones reported in 
literature for similar piezoelectric compositions. 
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Figure 13. a) Magnetostrictive properties (all] of the spinel plates and b] Piezoelectric loops (d33) of the PZT plates. 


The macro magnetoelectric sensors were fabricated by the addition of spinel and PZT plates 
in a laminate composite. To bond the plates, three different gluing conductive materials were 




















performed. The glues used were conductive paint, fired silver paint and conductive epoxy. 
The selected glue was conductive epoxy since the other presented additional problems. The 
magnetoelectric response of the sensors was characterized with two different circuits. First, 
the magnetoelectric response was measured by an open circuit and measuring the voltage 
with a high impedance voltimeter. The obtained results can be seen in the Figure 3. The 
voltage dropped due to leakage currents through the PZT plate. This was disadvantageous 
since the acquisition rate needed be higher, and this generated problems with calibration. 
An alternative circuit was to measure the current generated for the sensor and integrate it 
to obtain the polarization. The results obtained with this circuit can be seen in the Figure 14 
(right]. Here, the leakage currents were avoided since the PZT plate is short circuited. For 
both circuits, a high noise to signal ratio was detected. The noise mainly arose from the 
electrical network, because of its characteristic frequency of 50 Hz. The noise filtering needed 
improvement in order to improve the sensitivity of the device. 
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Figure 14. Left, voltage generated though the thickness of the sensor as a response of an applied magnetic field. Right: 
Polarization generated in the ME sensor due to application of a magnetic field. 


3.1.4 Useful Life Characterization of the Magnetic Sensor 

The useful life of the magnetic sensor was characterized through fatigue studies. This was 
made through the exposure of the sensor to an oscillating magnetic field with an intensity of 
0.5T. The polarization was measured as a function of the cycle number. The obtained results 
can be seen in Figure 15. A drop of 17% in the polarization can be seen when the sensor is 
exposed to 100 cycles. After this drop, the polarization remained constant until 14000 cycles. 
This pointed to the necessity of calibration of the device after a use period if a high sensitivity 
was required. At the same time, the sensor remained stable for a long exposure to high 
cycling magnetic fields, which allowed the application in a long useful life sensor. 
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Figure 15. Polarization as a measurement of cycle numbers. 

3.1.5 Pb(Zr,Ti)03 (PZT) Sol-gel Optimization 

The Pb(Zr,Ti)03 (PZT) sol-gel process was optimized to decrease the crystallization 
temperature. This was needed to decrease the lead evaporation and crack formation. The 
main problem in the PZT crystallization was the formation of pyrochlore phase; this phase 
decreased the piezoelectric properties substantially. The first approach was to add small 
amounts of water to produce internal hydrolyzation of the sol-gel. This work observed that 
the pyrochlore phase increased with the addition of water (this can be seen in the Figure 16). 
A second approach was to modify the hydrolysis temperature; here, temperatures ranging 
from 300°C to 450°C were evaluated. The optimum temperature was 400°C. As can be seen 
in the x-ray diffraction (XRD) spectrum in Figure 16 (left), the influence of the pyrolysis 
temperature with the pyrochlore phase formation was observed through the decrease in the 
100% peak for this phase at 20«29.75°. As expected, the amount of pyrochlore decreased 
substantially as the temperature of pyrolysis was increased. Finally, the time was modified 
for the crystallization temperature at 600°C. It was found that the optimum time for the 
pyrolysis was 120 sec with the large increase in the desired perovskite 100% peak at 
20«31.2°. Unfortunately, a small portion of pyrochlore was generated, and the composition 
needed to be modified in order to completely eliminate this phase before work could 
continue on the composite. 
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Figure 16. XRD characterization of PZT thin films with: left, different Hydrolysis rates and right, different Pyrolysis 

temperatures. 



















3.1.6 Additional 0-3 Composite Steps 

Regarding the 0-3 composite, CoFe204 (CFO] nano-particles were dispersed in 2- 
methoxyethanol. The nano-particles were synthesized using a hydrothermal process in an 
aqueous medium. Since the resulting nano-particles synthesized in the hydrothermal 
process were suspended in the aqueous solution, the nano-particle suspension could not be 
directly added to the PZT sol gel due the incompatibility of these two solvent systems. To 
replace the water from the suspension, a solvent exchange process was completed using 2- 
methoxyethanol. After three exchange reactions, the suspension was finally centrifuged and 
distilled. The obtained stable particles were added to the PZT sol-gel and spin coated. The 
spin coated film was characterized with XRD. In the XRD patterns, the PZT peaks could be 
seen, but there was presence of CFO peaks. This was due to the low concentration of CFO 
(10%] in the film. During processing of these films, it was found that the CFO nanoparticles 
began to agglomerate and fall out of suspension. Further work on the selection and control 
of the dispersants was needed to stabilize the particles. 

The first ferroelectric characterization was successfully completed for a PZT thin film. In 
order to complete these measurements, the optimal PZT films discussed above were 
deposited over a conductive wafer. The conductive wafer consisted of a Si wafer with several 
thin films, such as 500nm Si02, 50nm of Ti and 300nm of Pt. Over the PZT thin films, 
electrodes were deposited and patterned in order to have a precise upper electric contact. 
Finally, the film was characterized with a Radiant thin film ferroelectric tester; the obtained 
results did not show the characteristic piezoelectric loops; this was initially thought to be 
due to the presence of the pyrochlore phase. 

3.1.7 Preparation of Si02 Cantilever Process 

The Si02 cantilever process was established. A thermally grown Si02 wafer was patterned by 
photolithography with the cantilever pattern. After this process, the Si02 was dry etched 
with a mix of O 2 and CF 4 using the photoresist as a protective layer. Finally, the cantilever 
was released after a wet etching using tetramethylammonium hydroxide (TMAOH] solution 
in controlled conditions. 

After seeing the first results, it was noticed that the ratio between width and length was too 
small. The consequence of this was that poor mechanical properties were obtained. New 
attempts to obtain SiCh cantilevers were performed using smaller patterns with aspect ratios 
closer to unity, but problems with the release of the photoresist after the dry etching and 
issues with burned photoresist during the dry etch were encountered. The cantilever was 
released by a wet-etch using TMAOH in controlled conditions. The concentration of the 
etching solution was reduced, and this resulted in slower etching rates; but the reduced 
etchant strength permitted controlled release of the cantilevers with a low degree of 
structural degradation 

3.1.8 PZT Thin Film Fabrication and Optimization 

During this period, the PZT thin film fabrication was optimized and a final crack free, 
perovskite crystal structure PZT thin film was obtained. In order to achieve this, the chemical 
composition of the precursors was carefully characterized with the loss-on-ignition (LOI] 
technique. The results of the LOI characterization showed a discrepancy of up to 20% 



compared with the reported by the vendors. The finals were deposited on Si/Si02/Ti/Pt 
wafers obtained from a vendor. The results of this study can be seen in Figurel7. Figure 
17[left) shows a SEM micrograph of a crack-free PZT thin film over the Si/Si02/Ti/Pt layers. 
The thickness of the PZT thin film is ~300nm, and this was confirmed with ellipsometry 
(285nm). Regarding the crystal structure, the results of the optimized PZT thin film can be 
seen in Figure 17 [right). Here the crystal structure was characterized at different sintering 
temperatures for 60 sec. It can be seen that for temperatures higher than 650°C, a 100% 
perovskite phase was obtained; but for the 600°C thermal treatment, the non-desired 
pirochlore phase was present. With this study, the experimental conditions to obtain a crack 
free, perovskite phase PZT thin film were determined. 



Figure 17. SEM picture of Si/Si02/Ti/Pt/PZT thin film (left) and XRD characterization of PZT thin films annealed at 

different temperatures for 60sec (right). 


For the fabrication of the ME sensors, a process to deposit controlled layers of SiCh/Ti/Pt 
onto the Si wafers was needed. Different approaches were completed to investigate the 
effect of various processing conditions and thicknesses on the final piezoelectric thin film. 
The first approach was to deposit Ti and Pt via sputtering onto a thermally oxidized Si/Si02 
wafer. When the PZT thin film was deposited and annealed, the film suffered from various 
defect formations, such as delamination. This was due to several factors such as Si surface 
contaminates, in-plane strain related to Ti and Pt thickness, and strain relaxation processes 
during later thermal annealing processes. In order to resolve some of these issues, the 
following approaches were quickly evaluated: Si02 was grown on the Si surface by PECVD, 
intermediate annealing steps were implemented, analysis of cleaning process effects were 
monitored, and Ti and Pt thickness was modified. 


Regarding the 0-3 composite, initial experiments showed a high level of nanoparticle 
agglomeration. Various methods to re-disperse the obtained particles in 2-methoxyethanol 
were implemented, such as a short investigation of the use of PEI dispersants. 
Improvements were achieved, but the resultant dispersion was not optimal. The dispersion 
study can be seen in Figure 18; the addition of PEI dispersant permitted further dispersion 
of the agglomerated system, yet the primary particle size of ~20 nm was never achieved. 
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Figure 18. CFO particle distribution for different PEI concentrations. 


3.1.9 Ferroelectric Characterization 

In order to obtain a ME composite, a PZT thin film with good ferroelectric properties was 
needed. Because of this reason, the ferroelectric properties of the thin film were 
characterized and optimized. Several films were fabricated and Pt electrodes were 
deposited over them in order to have an upper contact. The process was optimized to achieve 
reproducibility and good ferroelectric properties. The results can be seen in figure 19. Here 
a good ferroelectric response without leakage currents, ferroelectric shape and remnant 
polarization. This process resulted in a reproducible fabrication of PZT thin films that was 
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Figure 19. Ferroelectric characterization of a PZT thin film deposited over Si/Si02/Ti0x/Pt wafer and annealed at 700°C 

for 60sec. 


3.1.10 Delamination 

The delamination problems previously found in the Si/Si02/Ti/Pt wafers were due to poor 
adhesion between the Ti and the Pt. This can be seen in Figure 20. In order to improve 
adhesion, the Ti was exposed to different oxidation processes. First it was oxidized in the 
RTA with the presence of air. The obtained film didn’t show good adhesion properties due to 
the formation of TiN in the surface. In order to avoid the formation of nitrogen the film was 
oxidized in an O 2 atmosphere outside the clean room. This showed better results but there 
were problems with particles contamination. In order to solve this problem a tube furnace 























was installed inside the clean room. This made it possible to oxidize the Ti thin films and to 
facilitate Si/SiCh thermal growth inside the clean room. Si/Si02 wafers were fabricated, 
excellent thickness distribution and roughness was obtained, and the obtained thickness 
was similar to the one predicted by diffusion equations. 
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Figure 20. SEM picture of Si/Si02/Ti/Pt thin film showing delamination in the Ti thin film. 

After this, different approaches were created. First, a thermal decomposition of Fe and Co 
acac was made using ethylene glycol as a solvent and oleic acid as dispersant agent. Several 
thermal decompositions of this system were performed but no particulates were obtained. 
This was due to the low temperature and low pressure of the synthesis. 

3.1.11 Solvothermal Synthesis of CFO Nano-particles 



Figure 21. Left, TEM image and right, XRD pattern of CFO nanoparticles. 


In order to have a better control of the solvothermal synthesis of CFO nano-particles, several 
modifications to the process were made in order to achieve good dispersion and retain the 
nano-size of the particles. As a result of this work, a new synthesis method for producing CFO 
nano-particles was developed using benzyl ether as a solvent, Co acac2 and Fe acac3 as a 
precursor. The dispersion system used was the 1,12 - dodecanediol, oleic acid and 
oleylamine system. As a result of this, well-dispersed nano-particles with the desired 






crystallography were obtained. As shown in Figure 21 (left), the particle morphology and 
size can be seen using TEM characterization. The TEM characterization confirmed the 
narrow size distribution and nano-size of the particles (~7nm). The crystal structure was 
characterized using XRD diffraction. As shown in Figure 21 (right), the XRD pattern can be 
seen showing the characteristic spinel structure and the crystallite size was calculated using 
the Scherrer formula, which confirmed a crystallite size of 6 nm. 

After the solvothermal synthesis of the nano-particles, the particles were dispersed in the 
PZT sol gel and spin-coated in order to obtain a particulate composite. Several approaches 
and new dispersants were used and finally a process was obtained to disperse the nano¬ 
particles in the PZT sol gel. This consisted of the aging of the PZT sol gel and the incorporation 
of CFO nano-particles in their hexane solution. The resultant solution with the nano-particles 
remained stable for more than two weeks. After this dispersion study, the thin films were 
deposited and annealed. These thin films did not show any cracks or delamination problems. 

The RCA cleaning procedure was successfully applied to clean the Si wafers from organics, 
ions and oxides. Various thicknesses for the Ti02 were studied and the influence with the 
adhesion of the Pt and final PZT crystal structure were analyzed. This study showed that 
more than 2 min (~100 nm) of Ti deposition were needed in order to achieve good adhesion. 
After this work, the influence of the thermal oxidation of the Ti with the texture of the PZT 
was studied. As can be seen in Figure below, the oxidation of the Ti layer produces a big (100) 
texture of the PZT thin film. 
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Figure 22. XRD patterns of PZT thin films with and without thermal oxidation. 

3.1.12 Optimization of CFO Nano-particle Dispersion in PZT Sol-gel 

In order to optimize the dispersion of the CFO nano-particles in the PZT sol gel, different 
studies were completed to control the surface characteristics of the nano-particles, and thus, 
manipulate the dispersion quality. First, the effect of the hydrolyzation of the PZT in the 
particle dispersion was studied. Here it was observed that the exposure of the PZT sol to 
humidity did not have a large influence on the particle dispersion. The effect of aging on the 
particle dispersion was also evaluated for different time periods. As it can be seen in figure 
1, the dispersion improves with aging within the solvent system. The surface peptization 
kinetics were limited. Here it was found that the particles could be successfully dispersed 
into the PZT sol but long periods of time to obtain complete dispersion would be needed. 
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Figure 23. Left, pictures of PZT sol with CFO nano-particles after; right, 2 days, middle 10 days, left 19 days. Right, particle 
size average after different periods of time. 

In order to decrease the required peptization time the sample was agitated for one day in 
order to investigate the improvement of the dispersion. The results obtained can be seen in 
Figure 23 (left). This did not improve substantially the peptization rate. As a second 
approach, different dispersants were used. As previously observed, the particle surface 
coordinates with -NH 2 groups and in order to be able to disperse the nanomaterials in 2- 
methoxyethanol with greater kinetics. With this in mind, dimethylformamide and 
diethylenetriamide were added to the PZT sol. The results obtained can be seen in Figure 
23 (right) The addition of dimethylformamide did not improve the dispersion, but the use 
of diethylenetriamide improved the dispersion drastically. This is because of the -NH 2 
terminal group and the polarity of the molecule. This gives an efficient way to prepare the 
PZT sol with CFO nano-particles well dispersed in a short period of time. Having a 
reproducible process to fabricate PZT sol with CFO nano-particles with different CFO 
loadings from 1 to 40% was fabricated. 



Figure 24. Left, pictures of PZT sol; left without shaking, right with shaking. Right, pictures of PZT; left without 
dispersants, middle with dimethylformamide and right with diethylenetriamide. 

With the above procedure, the first particulate composite was fabricated. In this case, a CFO 
loading of 2% was selected and the process described in Figure 25 was followed to obtain 
the composite thin film. This new deposition technique allowed the dispersion to higher 
concentrations of CFO nano-particles due to the gel coating of the particles. This resulted in 
films with low leakage current and good quality that were not previously able to fabricate 
with other processing techniques. 
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Figure 25. Modified sol gel thin film deposition. 

After the films were deposited, the quality was characterized using SEM imaging. The results 
can be observed in Figure 26. The figure shows that the thickness of the film remained 
constant after the addition of the CFO nano-particles. The quality of the films was not altered 
with the addition of CFO particles either. 

PZT thin film PZT with 2% CFO 



Figure 26. SEM characterization of; left PZT thin film and right PZT with 2% CFO loading thin film. 

The crystallography of the films was characterized at different annealing temperatures in 
order to determine the crystallization kinetics. It was found that with the addition of CFO 
nano-particles the PZT films crystallize without texture. This is due to the fact that the CFO 
particles worked as a nucleation sites, and in the case of the pure PZT films, the perovskite 
structure grows from the Pt wafer, as it can be seen in Figure 27. Another important 
difference is that the temperature required to obtain 100% perovskite phase is reduced from 
650 to 600 °C, without the formation of the non-desired pyrochlore phase. 

















Pt 

1) 


(200) 700°C 

1 fct_ 

1 ^ 

a 650°C 

It 

600°C 


+ Pyrochlore 


( 100 ) 

JL. 


JL 


( 110 ) 

JL 


JL_j J 

JL_# 


30 40 

Angle (2 Theta) 


50 



Figure 27. XRD characterization of; left PZT thin films sintered at 600, 650 and 700 °C and right PZT films with 2% CFO 

loading sintered at 500, 600 and 700 °C. 


Finally, the ferroelectric properties of the films were characterized using the Radiant tested 
whis is depicted by Figure 28. The addition of CFO nano-particles to the films did not 
generate any leakage currents due to the good dispersion of the particles. In addition, the 
incorporation of nano-particles increased the remanent polarization and the coercive field, 
making the films behave more like a hard piezoelectric. 
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Figure 28. Ferroelectric characterization of PZT films; in red without CFO and in black with CFO nano-particles. 

This new technique to fabricate particulate composite thin films provided an effective 
synthesis to obtain films with high particle concentrations and good dispersion giving as a 
result a higher ME response. This enabled fabrication of micro-magnetometers with high 
response in a Si based circuit that could be easily integrated. 

3.1.13 Micro-Chemical Sensor Array 

Photo-lithography and physical vapor deposition processes were developed to produce 
singular micro-chemical sensors on resistive alumina substrates. Fabrication of the micro¬ 
sensors was initiated through the development of processes to pattern gold (Au) 
interdigitized electrodes (IDE) onto alumina substrates. The initial IDE geometry targeted 
were four sensors array in a road. The dimensions for the different sizes in the mask are 
outlined in the following chart. All these dimensions are in microns and correspond to Figure 
29. 
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Figure 29. IDEs geometry. 
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The Au-IDEs were patterned through a basic lift-off process. A Larurell Technologies 400 
Series spin-coater was used to deposit AZ5214 photoresist at a thickness of ~1.5 pm onto 
the alumina substrates. The AZ resist was then baked on the samples for 5 min at 95°C on a 
hotplate before being placed in a Suss Microtec MAG mask aligner. The mask aligner then 
exposed the wafer to UV light through the mask, cross linking the polymer. This model 
operates at UV wavelength energy of 100 MJ/cm 2 . The intensity of the light was measured 
(in MW/cm 2 ) before each exposure so the exposure time could be determined. After 
exposure, the wafers were developed using AZ400K developer in a quick immersion. The 
developer removed the exposed resist and created a void in the shape of the IDE. Following 
the patterning of the resist, a CVC 610 DC Magnetron sputter station was used next to deposit 
Au over the entire alumina wafer. First, a titanium coating was deposited onto the wafer to 
assist with adhesion of the metal to the alumina substrate. A layer of Au was then deposited 
under an argon plasma flow at 570 volts and 6.0 Torr for 180 seconds. A lift-off process was 
performed as follows: The substrates were immersed in an acetone bath for 20 minutes and 
periodically agitated with ultrasonication. This removed the polymer mold and any metal on 
its surface. The samples were then placed in a methanol bath for 5 minutes, cleaned with de¬ 
ionized water, and dried with nitrogen. Two Au IDEs can be seen in Figure 30. 




Figure 30. AU IDEs. 


3.1.14 Multi-step Hydrothermal-Assisted Procedure 

The nano-scale CuO pattern was created using a multi-step hydrothermal-assisted 
procedure. First, alumina substrates with fully developed IDEs were spin coated with AZ 
5214 photoresist, using the same spin rates. They were exposed under a negative sodalime 
photomask from in the Suss Microtec MAG mask aligner. The pads were aligned with the 
IDE’s on the substrate by using a microscope to align the crosshairs on the substrate (made 


















































with the IDEs) with the squares on the mask. Next, the substrates were agitated in the 
AZ400K developer during a quick immersion. This process created a mold of rectangular 
pads over the IDE fingers. The dimensions of these pads were proportional to the length of 
the IDE that they were matching. 

The next step in the process was metal deposition via sputtering. The substrates were coated 
with a layer of Cu using the same sputtering station as in Part 1, with an argon plasma flow 
at 490 V and 6.0 Torr for 6 seconds to seed the surface. Then a liftoff process was performed 
again, to reveal Cu micro-pads on top of the micro IDEs. 

The alumina/IDE/Cu samples would undergo a templated hydrothermal process to grow 
CuzO/CuO nanorods on the Cu pad. This was accomplished by forming a highly basic solution 
(ph of 11.1) of copper nitrate hydrate (Cu(N03)2) and de-ionized H 2 O via precipitation with 
ammonium hydroxide. The solution and alumina/IDE/Cu sample were held in a stainless 
steel autoclave at 100°C for 1 hour. The powder was characterized using XRD and Cu20/Cu0 
traces were found. 

3.1.15 Hydrogen Detection Micro-chemical Sensor 

A micro-chemical sensor for the detection of hydrogen in humid air was successfully 
developed. The chemical sensor was constructed on an alumina substrate. The IDEs were 
made of platinum using a lift-off process. The nano-CuO sensing material was initially 
patterned onto the alumina substrate and Pt IDEs with a sputtering process. Nano-wire CuO 
sensing structures were then synthesized through a controlled thermal oxidation process 
developed over the last month of work. 

The micro-sensors were tested in flowing hydrogen for 10 minutes at different 
concentrations ranging from 500 to 5000 ppm H 2 . The sensors were allowed to recover in 
air for 20 minutes. During the second third of the test, the sensors were tested in the same 
concentration conditions, but a faster response was tested by flowing hydrogen gas for only 
1 minute and a 20 minute recovery time. In the last part of the test, the fast response of the 
micro-sensors was tested at a high concentration. Results indicate that the sensors are able 
to detect high concentrations of hydrogen at times as short as 5 seconds. Test results are 
shown in Figure 31. 



Figure 31. Test results for a hydrogen chemical sensor at 340 Q C. 






















A micro-chemical sensor utilizing large grained CuO sensing materials was developed by a 
sputtering/oxidation process. The IDEs were made of platinum using a lift-off process. The 
CuO sensing material was patterned onto the alumina substrate by first DC sputtering the Cu 
across the Pt interdigitized electrodes (IDEs). The Cu metal layer was then oxidized in a 
controlled environment to produce the continuous CuO layer across all of the electrodes. 
The resultant microstructure showed CuO grains with little porosity through the 
microstructure. This indicates that the active surface area for gas reaction was very low 
resulting in a lower than desired sensing rate. It is well known that by increasing the 
available surface area for gas interaction with the electrode would result in higher sensitivity 
and sensing rate. 


A hydrothermal process for forming nano-CuO particles was developed. The nano-CuO 
material was then incorporated into an ink formation, and this formulation was painted onto 
the IDEs previously developed. In addition, a micro-casting process for forming distinct 
shapes over the IDEs was also developed and utilized to micro-print the nano-CuO absorbent 
material into the desired locations. Figure 32 shows a picture of the final micro-sensor 
platform after the final processing step. Figure 33 shows a four sensor array built to test H 2 , 
CO, NO, and NO 2 at the same time. In this case, the lead test wires were attached to the second 
sensor to test H 2 content within a gas stream. 


Figure 34 shows the resultant test data from a representative micro-sensor produced with 
the above discussed processing methods. This sensor was different from those tested 
previously, in the case that these contained the desired nanomaterial CuO. The micro- 
sensors were tested in flowing hydrogen for 10 minute exposures at different concentrations 
ranging from 1000 to 4000 ppm H 2 (mixed with N 2 ). The sensors were allowed to recover in 
air for 20 minutes. During the second third of the test, the sensors were tested in the same 
concentration conditions, but a faster response was tested by flowing hydrogen gas for only 
1 minute and a 20 minute recovery time. In the last part of the test, the fast response of the 
micro-sensors was tested at the 4000 ppm level for very short exposure times. The intent of 
this test was to identify the maximum response rate of the sensor. The figure shows that the 
sensors were able to detect high concentrations of hydrogen at times as short as 30 to 1 sec. 
The response shows that a similar diffusion process was required in each case to alter the 
sensor surface back to the baseline steady state condition. 
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Figure 32. Picture of a singular micro-sensor with CuO nanomaterial absorbents across the IDE pattern. 






Figure 33. Picture of a four element array of nano-CuO micro-sensors (with one sensor connected through contact via). 



Figure 34. Sensing performance of a singular micro-sensor exposed to different low levels of H2 for times ranging from 10 

min to 1 sec. 

3.1.16 Fabrication of Micro-chemical Sensors With CuO 

Micro-chemical sensors were fabricated utilizing CuO as a sensing material. These sensors 
may be divided in three groups which differ in the manner that the CuO is deposit over the 
interdigitized electrodes that all of them share. 

These are as follows: 


















1. Thermally grown CuO 

2. Hydrothermally grown CuO nanoparticles 

3. Painted with nano-CuO ink 

To make the electrodes, the circuits were patterned by photolithography and then an 
adhesion layer of Zr and a conductive layer of Pt were sputtered over the photoresist. After 
a typical lift off process, the IDEs of Pt were obtained. 

In the first group of sensors, a layer of Cu was sputtered over the IDEs and oxidized to obtain 
CuO. This method is good because of its simplicity. Previously, the microstructure of the CuO 
obtained was very dense with grains on the order of micrometer. The direct consequence of 
this low active surface was on the decrease of sensibility to chemical gases. In order to 
overcome this problem, nano-needles of CuO were successfully grown on a copper foil. This 
result encouraged us to recreate the same conditions over a layer of Cu sputtered. Although 
nano-needles were not obtained the grain size was reduced. 

3.1.17 Re-creation of Hydrothermal nano-CuO Sensors 

Sensors with hydrothermally grown nano-CuO were re-recreated. In order to do this, the 
wafers with the IDEs were sputtered with Cu (over a feature patterned by photolithography]. 
The samples were placed in the autoclave horizontally with a basic solution of Cu(N03]2-H20 
and washed in order to obtain the desire pattern. The advantage of this method relies on the 
possibility to obtain CuO nano-plates that grow on the Cu. The first results showed that the 
CuO had grown, but not continuously. The CuO was well attached to the Pt electrodes but 
between them a bridge was not created. In order to overcome this situation, the thickness of 
the sputtered Pt was reduced to lOOnm. The problem of reducing the thickness of Pt is 
related to the reduction of resistance. A study showed that when Pt film thickness exceeded 
100 nm, further Pt deposition had no significant effect on the conductivity improvement. 
After repeating the hydrothermal growth with the new thickness, the results showed an 
improvement on the continuity of the CuO. In spite of this encouraging result, the sensor 
sensitivity was lower than desired. 

The third group of sensors was fabricated using the CuO nano-plates created during the 
autoclave run mentioned before. The idea of this method is very simple. One stencils a CuO 
over the IDEs by painting them with ink created with the CuO. After the deposition, the 
sensor is submitted to a temperature cycle in order to eliminate the organics. In the 
beginning, the created ink did not allow us to obtain a continuous CuO layer. After several 
attempts, the composition of the ink was changed allowing us to obtain a better control 
during the deposition (since the viscosity was increased], and also it allowed us to get a 
continuous layer of CuO over the IDEs. 

3.2 Subtask 2.2: Ultra-Low Power Analog Event Detection 

The objective of this research direction was to develop techniques for building ultra-low- 
power hardware event-detection systems. The goal was to create dedicated "wake-up” 
circuits for remotely deployed sensor nodes so that the portions of the sensor node that 
consume proportionally large amounts of power could be turned off when not needed. As 



such, the use of low-power analog circuitry was investigated, that consumes only a few 
micro-Watts to be used as an energy-management tool to enable the high-power-consuming 
subsequent digital systems only when action is required. Additionally, this analog circuitry 
was able to provide initial pre-processing of the sensor information to pre-condition the raw 
sensor data and provide feature extraction. 

Analog systems were specifically investigated for acoustic and vibration sensing systems. 
For example, Figure 35 illustrates work, which focused on developing a custom integrated 
circuit that was able to perform spectral decomposition of incoming sensor data, which is 
analogous to the commonly used digital fast-Fourier, transform in a wide variety of signal¬ 
processing systems. This integrated circuit, which was fabricated on a standard 0.5jum CMOS 
process, consisted of two filter banks and sub-band magnitude detectors that perform 
spectral decomposition for use by the subsequent classifier. Classification is performed with 
double-threshold quantization in each band followed by a programmable logic array (PLA) 
which compares the spectral inputs with stored detection rules. The PLA stored spectral 
templates which were used to match incoming signals to their respective classes. Once 
specific classes were observed, the system generated an interrupt signal that wakes up the 
digital system from a state of hibernation. Figure 35 also includes details on how we the 
custom integrated circuit was interfaced with commercial off-the shelf wireless sensor 
network nodes, called motes. 
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Figure 35. Left, block diagram of the analog event detector, showing how it interfaces with sensing nodes. Right, 

demonstration of the event detector 1C. 

Also included Figure 35 is a demonstration of the event detector integrated circuit. In this 
particular case, the input was a logarithmic chirp with tone bursts at 200Hz and 635Hz, 
corresponding to channels four and nine (shown in the third and fourth subplots). The 
second subplot shows the measured magnitude outputs in spectrogram form. The PLA was 
programmed to perform an AND of those channels, and correctly outputs a "high” (i.e. wake- 
up signal) when both tones are present. 



























































































3.2.1 Development of Vehicle Classification System 

To evaluate the event detector in the context of a remote sensing application, a vehicle 
classification system was developed. A dataset consisting of forty 10-second recordings of 
cars and trucks measured by a microphone was used. The objective was to detect the 
presence of a vehicle and classify it as a car or truck. Training software was implemented 
which used half of the recordings to learn the comparator threshold levels and PLA detection 
rules that best classify an input signal as either a car, truck or no vehicle. The event-detection 
integrated circuit was tested by streaming the other half of the vehicle recordings from a 
digital-to-analog converter into the filter bank input. Two PLA output pins were used, 
including one to indicate vehicle presence and one to indicate car or truck. The mote’s 
microntroller woke up when the "vehicle presence" pin was asserted. Upon waking, the mote 
implemented a state machine, monitoring both of the PLA pins and using the sequence of 
outputs from the PLA to make the final decision. Accuracy and power consumption values 
are provided in the following Tables, and the power projection plot shows the power 
consumption of an all-digital implementation, our first system [called "w/ ASP”], and our 
newer implementation ("Projected Power"). As can be seen in Figure 36, for a low frequency 
of events, our complete ASP system combined with a digital mote can be expected to 

continuously operate for approximately 9 years off of two AA batteries. 

Specifications 


Vehicle Classification Results 



Ground Truth 


NULL 

Car 

Truck 

Missed 


20% 

0% 

Car 

0 false alarms 

70% 

10% 

Truck 

0 false alarms 

10% 

90% 


Specification 

Value 

Device 

Power 

Chip Area 

2.25mm 2 

Filter bank 

1.49/xW 

Channels 

16 

Comparators & PLA 

10.5/xW 

Chip Power 

27.8/xW 

Read-out buffering 

15.78/x W 

Supply Voltage 

3V 

External biasing 

18.9/xW 

Technology 

0.5^ m 

ASP + Biasing 

46.7//W 

Frequency Range 

20Hz-100kHz 

Mote (sleep) 

78/xW 

Dynamic Range 

50dB 

Mote (awake) 

1.5mW 



Mote (transmitting) 

60mW 


Figure 36. Vehicle Classification results and Chip Specifications 


3.2.2 Development of Motion Detection Systems 

Research was conducted toward the design of motion detection systems using custom CMOS 
imagers. We demonstrated a one-dimensional motion detection system based upon the 
visual system of a housefly that was able to wake up a larger camera network in the presence 
of motion in a specific direction. This analog motion detector operated in the 10s of 
microwatts of power. To supplement the operation of this motion detector, we designed a 
speed-detection circuit that was able to determine how fast an object is moving in a specific 
direction. The speed estimate can be combined with the biologically inspired motion 
detection system to provide further information of the object under view when the wakeup 
signal is generated. 


When developing these analog circuits, dynamic range was an important consideration, and 
ultra-low-power analog circuits typically had extremely small linear ranges that limit the 
largest signal that can be operated upon. As a result, a technique was developed to increase 
the linear range of operational transconductance amplifiers, which are a basic building block 













of low-power analog circuits. This technique used an effective adaptive biasing by using 
floating-gate transistors, which are the core element of digital Flash memory). These 
traditionally digital memory elements were used for analog applications, and were able to 
increase the linear range of operational transconductance amplifiers by a factor of 16 or 
more. 

Floating-gate transistors were also used to help extend the dynamic range of analog circuits, 
which was an extremely important consideration for low-power and low-voltage circuits. As 
a result, a technique was developed to increase the linear range of operational 
transconductance amplifiers, which are a basic building block of low-power analog circuits. 
This technique used an effective adaptive biasing by using floating-gate transistors. Figure 
37 illustrates the extension of the linear range of operational transconductance amplifiers. 
By adjusting the programmed charge on the floating gate, the linear range of these amplifiers 
were increased (as indicated by a straightening of the line in the left-hand plot) which can 
be seen in Figure 37. Figure 37 also depicts the tradeoff of a reduction in the 
transconductance of the amplifier (as shown on the right), but the transconductance became 
more constant over a larger range of voltages, making this circuit easier to use in many 
situations. 



Figure 37. Increased linear range of amplifiers (left] and reduction in the trans-conductance of the amplifier (right]. 


3.2.3 Reverse Tunneling 

Considerable progress was made in identifying the causes of certain of the non-idealities 
(including something called "reverse tunneling” that causes the charge on the floating gates 
to be less than the desired/programmed amount). Ways to better isolate the floating-gate 
transistors to remove this unwanted effect were investigated. Performance of two types of 
"tunneling junctions” in terms of speed of programming and charge retention were also 
perfromed. 


Investigations into building ultra-low-power short-term memory that did not require going 
through a high-power analog-to-digital converter and then being stored in some sort of 
conventional memory system (e.g. RAM or Flash memory) were performed. The theory was 
that if a wake-up detector was used to find an important event, then all information leading 
up to that event would be lost (since the higher power system only wakes up once an event 









has been seen]. This missing information was a problem with any type of wake-up detector. 
Ways were sought to temporarily store analog information without the cost associated with 
converting it into the digital domain. A proof-of-concept peak detection system was built 
that shifts select information of a signal into an array of capacitors to hold the data for a short 
period of time. This circuit effectively sampled the signal at select times and conveyed that 
amplitude value and the timing value to the capacitor array (i.e. the short-term memory). 


3.2.4 Development of Analog Signal Processor 

A new programmable analog signal processor (ASP) was developed that was capable of 
performing spectral decomposition, matching spectral content to a "trained” set of templates 
for classification purposes, and then waking up a subsequent digital system when an event 
of interest has been identified. To verify the performance of the new ASP architecture, we 
used our ASP to perform vehicle classification while the digital portions of the sensor node 
were in sleep mode to save power. When the ASP detected the presence of a vehicle, the ASP 
woke up the microprocessor and antenna for further processing and communication, 
respectively. The microprocessor, once awakened, was used to provide further refinements 
to the classification of the sensed phenomena. The power savings of using the new ASP core 
were significant. A fully digital implementation performing the same task could only survive 
on a single battery for only up to a few months. Using the ASP allowed the system to remain 
on for a projected 7-9 years. Figure 38 illustrates the architecture of our ASP as well as the 
projected power savings. 


Analog Event Detector 




Figure 38. Left: block diagram of our analog event detector , showing how it interfaces with sensing nodes. Right: comparison 
of the power consumption of our ASP approach versus all digital vehicle classification systems. 


This new ASP required significant improvements to the infrastructure to provide for 
programmability, reconfigurability, accuracy, and precision. The most significant 
improvements to the infrastructure were made by introducing floating-gate transistors to 
serve as programmable analog elements. During this course of this project, the following key 
advances in floating-gate transistor infrastructure were made, including (1) a fine-grain, 
high-accuracy programming algorithm using Matlab in the loop, (2) the creation of a high- 
voltage charge-pump to generate the voltages necessary for programming on the chip (to 
reduce external circuitry requirements), (3) implemented a simulation model for floating- 
gate transistors within the Cadence simulation environment, and (4) the initial development 

















































































of basic "self-programming” floating-gate circuits. In addition to creating this new necessary 
infrastructure, considerable time was spent characterizing floating-gate transistors across 
multiple processes in order to better understand the device physics related to long-term 
retention and the most power-efficient programming techniques. 

This circuit effectively sampled the signal at select times and conveyed that amplitude value 
and the timing value to the capacitor array (i.e., the short-term memory]. The result was that 
the sampling rate was adapted as the frequency content changed so that the signal was 
constantly sampled at the "instantaneous" Nyquist rate. Since sampling was at irregular time 
intervals, considerable time was spent determining how to reconstruct the original since the 
typical Shannon-Whittaker reconstruction method (i.e., scaled and shifted sine pulses) did 
not apply to this system. Three completely unique reconstruction methods were developed 
(asymmetric sine functions, asymmetric sine-squared functions, and Bezier polynomials) 
that perform reasonable reconstruction. Figure 39 shows the reconstruction of a speech 
waveform. Traditional Nyquist-rate sampling would require > 10,000 samples, and the 
adaptive sampling system compresses the data into only 652 samples, all while consuming 
slightly over lpW of power. 


(a) Amplitude Modulation 


(c) Speech Signal 
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Figure 39. (a-b) The system is capable of capturing a wide range of amplitudes and frequencies. For these plots, the input is 
shown in grey, detected max/mins are black dots, and the reconstructed wave is a black dashed line, (c) The speech clip used 
as input to the system is shown in grey with detected max/mins in black dots, (d) The speech clip approximation 
from using the modified Bezier formula with the sampled max/mins. 


This work began with populating the new printed circuit boards (PCBs) that were designed 
and verified. With those PCBs properly working, the testing of the integrated circuits was 
initiated. This integrated circuit contained a large number of stand-alone test cells as well 
































































































as several integrated systems. The major circuits and systems consisted of (1) floating-gate 
transistor characterization circuits and infrastructure, (2] reconfigurable analog arrays, and 
(3) an improved version of the short-term analog memory buffer that was prototyped. All of 
the test cells and systems have been tested and their operation has been verified; all circuits 
and systems operate as designed. 



Figure 40. The integrated circuit. 



Figure 40 depicts a top view of the integrated circuit. Figure 41 depicts the PCB, while Figure 
42 displays the results of the integrated circuit [one component of the analog sampler 
system.) 


Figure 41. The PCB 
















3.3 Subtask 2.3: Compressive Sampling and Low-power Collective Sensing 



Figure 43. Illustration of multiple-description coding (MDC) for two channels and three receivers (Decoder 0 - central 

receiver; Decoder 1 or 2 - side receiver}. 

Two complementary directions were explored for robust image transmission through 
sensor networks: multiple description coding (MDC) and error-resilience coding (ERC). 
MDC was originally proposed for error resilient transmission of images in communication 
networks. It represented an alternative approach toward exploiting diversity to fight against 
channel errors to forward error correction (FEC). Figure 43 shows a general diagram for 
transmitting a source (e.g., an image) through two channels (i.e., generate M=2 descriptions). 
Depending on the network condition, two kinds of decoding conditions at the receiver could 
be observed: 1) both descriptions are successfully received which is often called central 
receiver; 2) one description is lost during the transmission, which is often called side 
receiver. 

A key innovation was the nonlocal regularization (which was previously exploited for 
concealing lost blocks) can be combined with an arbitrary number of observation constraint 
sets (i.e., successfully received descriptions). This was a natural extension of previous work 
on maximum a posterior (MAP) decoding from point-to-point communication to network 
communication. Apparently, if the two-bit streams (descriptions) contain identical 
information, nothing could be gained. As long as they were complementary to each other - 
e.g., one was the translated version of the other one, potential gain existed (note that wavelet 
transform used by SPIHT is NOT translation invariant). A maximum-likelihood (ML) 
decoding enhances the reconstruction result by taking the average of two (the PSNR gain 
over single-description is around ldB). It was found that if the ML was replaced by MAP 
decoding, another ldB gain could be achieved and visual quality improvement was 
noticeable too (see Figure 44). 






























Figure 44. Top: decoded image at the side receiver (PSNR1 = 26.42dB, PSNR2 =26.24dB) by SPIHT at the bit rate of 
O.lbpp. Bottom: decoded image at the central receiver by different algorithms: left - ad-hoc averaging of two decoded 
versions (PSNR=27.30dB); right - MAP decoding from two received bit streams (PSNR=28.36dB). 

3.3.1 Error Resilient Image Coding. 

A joint TIP paper [5] was written which extended current state-of-the-art image coding 
SPIHT from error-free to error-prone situations. The basic idea was to replace the 
conventional significance map by a new data structure named progressive significance map. 
The new data structure was less sensitive to random channel errors because it prevented 
catastrophic error propagation. The preliminary experimental results [as shown in Figure 
45) justified the superiority of error-resilient SPIHT in error-prone environments. 
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Figure 45. Rate-Distortion performance comparison between conventional (old) and error-resilient (new) SPIHT 

algorithms on Lena image. 


















Work was performed on extending our previous work on collective sensing [6] and nonlocal 
sparsity-based image recovery [7] into error concealment applications. The original work 
was based on the assumption that some blocks were missing but the remaining are 
distortion-free. In real-world scenarios, such assumption is seldom satisfied because of the 
quantization distortion introduced at the encoder. Therefore, the quantization constraint 
could be handled by borrowing ideas from [6] where it has been shown how nonlocal 
sparsity helps lossy image decoding. The experimental results (as shown in Figure 46) have 
shown some promising improvement over previous work. 



Figure 46. Error-concealment performance comparison between morphological component analysis (MCA)-based (left, 
PSNR=22.17dB) and ours (right, PSNR=24.30dB) on lena512 image [6]. 


3.3.2 Open-source Implementation of SPIHT Image Coding. 

After meeting with Trident personnel and discussing their immediate need, it was agreed 
that exploration to the feasibility of using wavelet-based SPIHT algorithm to compress 
images acquired by the sensor node was necessary. The SPIHT algorithm admitted an open- 
source implementation called Qccpack that was developed and tested on Feroda Linux. 

3.3.3 Integrate Image Codec at the System Level. 

Image compression as located at the application layer needed to work with other 
applications as well as network and physical layers in practical transmission scenarios. On 
one hand, the connection of image compression with high-level vision tasks such as object 
detection and tracking was explored. On the other hand, there was a necessity to understand 
how the image source matched with the characteristics of channel. In an ad-hoc network 
environment, optimization of image transmission at the network level seemed an under¬ 
investigated topic and worth further research. 









3.3.4 Error Concealment for Multiple-description Coding. 

The idea of multiple description coding (MDC) was to exploit diversity (e.g., multiple 
antennas at the transmitter or receiver) as a means of fighting against chair impairment (e.g., 
fading). In fact, many error concealment techniques were also based on exploiting diversity 
(redundancy) with the image source such as self-repeating patterns of textures. 

3.3.5 Low-rank Approximation for Simultaneous Sparse coding. 

Work was performed on the problem of joint dictionary learning and group sparsity 
optimization (a.k.a. simultaneous sparse coding). Motivated by the success of low-rank 
methods in matrix completion and denoising, novel image denoising algorithm was 
developed called low-rank approximation simultaneous sparse coding (LASSC) [8]. 

The concept of low-rank approximation has also been generalized to facilitate the 
compression of multiple images and has also been generalized into image interpolation, 
inpainting and compressed sensing. A journal article [9] and two conference papers were 
submitted related to this line of research. 

3.3.6 Local Self-examples for Image Resizing and Super-resolution. 

Inspired by the recent advance in video/image upsampling from local self-examples [10], 
other potential applications were explored, such as content-aware resizing of satellite 
images and resolution enhancement of face images. Such local self-similarity could also be 
exploited by image compression algorithms tailored for a specific class of images (e.g., 
satellite or face). 

3.3.7 Network-centric SPIHT Image Coding. 

Based on previous work [5], new frontiers of combining error-resilient SPIHT with multiple- 
description coding (MDC) and network coding were explored. The basic idea was to jointly 
exploit the diversity at both the source (e.g., multiple and progressive descriptions) and 
channel (e.g., physical and network layer) sides. How to accommodate the redundancy and 
discrepancy arising from a heterogeneous network scenario (e.g., images codes by JPEG and 
SPIHT respectively) was new research topic that has not been well understood. The goal 
was to demonstrate some initial gain over previous work on networking image coding for 
multicast [11]. 

Work was performed on extending current state-of-the-art image coding SPIHT algorithm 
from error-free to error-prone situations. The error-resilient SPIHT algorithm was based on 
the idea of introducing progressive significant map and designing fixed-length codes for 
location-related source information. It was experimentally verified that the new error- 
resilient SPIHT algorithm outperforms the original in an error-prone environment. This line 
of research produced one journal publication [5]. 

Investigations on the idea of combining error-resilient SPIHT with multiple-description 
coding (MDC) and network coding were performed. In order to jointly exploit the diversity 
at both the source (e.g., multiple and progressive descriptions) and channel (e.g., physical 
and network layer), it was proposed to study a biologically inspired representation of data 
by their associative relationship. 



Moreover, the idea of network-aware image coding could be further generalized into 
combining the strategy of multiscale source representation [redundancy at the source) with 
the self-similar topology of network [redundancy of the channel). Unlike existing networking 
coding for P2P streaming, focus was placed on a theoretical understanding of how to 
transport a single source that is hierarchically organized [e.g., natural images) to multiple 
destinations, which can be viewed as a multiscale generalization of multiple-description 
coding [MDC) problem for broadcasting channel. The initial study with toy examples showed 
that it is possible to reorganize the representation of source to facilitate the information flow 
through a hierarchically-organized network [e.g., internet whose traffic demonstrates the 
character of self-similarity). 

3.3.8 Joint Source-channel-network Image Coding. 

The idea of combining error-resilient SPIHT with multiple-description coding [MDC) and 
network coding was investigated. In order to jointly exploit the diversity at both the source 
[e.g., multiple and progressive descriptions) and channel [e.g., physical and network layer), 
it was proposed to study a biologically inspired representation of data by their associative 
relationship. Such associative data representation admits a metric-space analysis and had 
the potential of matching the hierarchical structure of communication networks. The next 
goal was to experimentally demonstrate some initial gain over previous work on networking 
image coding for multicast [12]. 

Extending the work on progressive significance map was published in the teams recent work 
[5]. The basic idea was to combine the strategy of multiscale source representation 
[redundancy at the source) with the self-similar topology of network [redundancy of the 
channel). Similar idea was studied in [13] for P2P streaming; while outs focused more 
hierarchical extension of network coding from a theoretical perspective [e.g., the 
generalization of original butterfly network to a multi-scale topology). 

Unlike [12] for P2P streaming, focus was placed on a theoretic understanding of how to 
transport a single source that is hierarchically organized [e.g., natural images) to multiple 
destinations, which could be viewed as a multiscale generalization of multiple-description 
coding [MDC) problem for broadcasting channel. The initial study with toy examples showed 
that it was possible to reorganize the representation of source to facilitate the information 
flow through a hierarchically-organized network [e.g., internet whose traffic demonstrates 
the character of self-similarity). 

3.3.9 Image Compression for Pattern Recognition. 

Work on the development of recognition-targeted image compression algorithms [e.g., 
better preserve the feature useful to pattern recognition than intensity values) was 
performed. 

3.3.10 Low-rank Approximation for Computational Imaging. 

Motivated by the success of low-rank methods in matrix completion and denoising, a novel 
low-rank solution to the problem of simultaneous sparse coding [LASSC) was developed [9, 



12,14]. On theoretical side, a new bilateral variance estimation interpretation was obtained 
and on the practical side, its application into multispectral imaging was explored. 

3.3.11 Low-rank Methods for Image Restoration. 

Motivated by the success of low-rank methods in matrix completion and denoising, a novel 
image denoising algorithm based on spatially-adaptive iterative singular-value thresholding 
was developed [12]. On the theoretical side, a new bilateral variance estimation 
interpretation was obtained and on the practical side, its applications into multispectral and 
low-illumination imaging was explored. 

Motivated by the success of low-rank methods in matrix completion and denoising, the 
previous work into image restoration, superresolution and compressed sensing was 
extended [9, 13, 14]. Two new NSF proposals were developed and submitted to NSF CCSS 
and CIF programs respectively. 

3.3.12 Source Coding for Pattern Recognition. 

The definition of image quality heavily depended on the application scenario (e.g., image 
coding vs. pattern recognition). A high-quality JPEG compressed iris image with severe 
eyelid occlusion could have little merit from the recognition point of view. It was desirable 
to extend the previous work [14] into a network-aware situation - i.e., to understand the 
impact of network transmission uncertainty on recognition performance. 

Recognition-targeted image compression algorithms were developed (e.g., better preserve 
the feature useful to pattern recognition than intensity values). 

Such line of research was inspired by the success of sparsity in both image compression and 
pattern recognition - how to obtain a sparse yet redundant representation of images to 
support both low-level and high-level vision. The long-term objective along this line of 
research was to gain a deeper understanding of the connection between two seemingly 
distant fields - source coding vs. pattern recognition. The underlying hypothesis was that a 
generalization of vector quantization in source coding could shed novel insight to the 
problem of pattern recognition (e.g., binary classification can be viewed as a 1-bit 
quantization problem). 



4 Task #3: Design, test, and demonstrate real-time fault-tolerant 
protocols for data aggregation, inference, location tracking, and 
connectivity maintenance in a mobile heterogeneous sensor 
network. 

4.1 Subtask 3.1: Distributed Coding and Data Exchange in Heterogeneous Sensor 
Networks 

The problem of physical field estimation produced by an object or subject using sensory data 
collected by a distributed sensor network was a challenging problem. While the problem 
was approached by researchers working in the field from a variety of perspectives [15,16], 
in its general form it was not been fully solved. Its general statement requires involvement 
of general models as well as general constraints on power and bandwidth. The teams 
approach initially involved basic simplified models and constraints and relied on classical 
estimation theory. 

4.1.1 Unknown Object Location and the Strength of Physical Field 

It was assumed that distributed network of heterogeneous sensors acquired samples of a 
physical field. Consider, for example, a distributed network composed of magnetometers, 
which senses a magnitude of magnetic field generated by some ferromagnetic object 
positioned in a vicinity of sensors. A ferromagnetic object can be modeled as a simple dipole 
generating a magnetic field with the field decaying as the inverse cube of the distance to the 
dipole. In a typical sensor network, the location of the dipole is not known as well as a type 
and size of the object, which was characterized by the strength of the magnetic field. It was 
further assumed that the sensor network was calibrated in the sense that the relative 
locations of the sensors distributed over some area were known. Sensors act independently 
of one another and take noisy measurements of the physical field. If the bandwidth of the 
network is constrained, each sensor would have to encode its data and communicate 
encoded data to the fusion center for a join field reconstruction. In this initial work, it was 
assumed that both sensory noise and the noise in communication channels was white 
Gaussian with known statistics, that channels do not interfere and that the encoding is 
performed through deterministic quantization by using a completely specified M level 
quantizer. 

Mathematically, this problem can be formulated as follows. Assume that a network of K 
sensors covers a known area A. Let be known reproduction points of a least-square 

quantizer with M levels. Then the join log-likelihood of the observed sensory data at the 
fusion center, ,..., z K , is 

Equation 1 
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where p k j are the probabilities for the output of the sensor K to be mapped to the j-th 
reproduction point during the encoding process 

Equation 2 


Pk, 


L j +1 

■j- 


Vi 


exp 


71(7 


(f-G k ) 

2<j 2 


2 \ 


dt, 


<7 2 and 77 2 are the variance of the sensory noisy and the variance of the Gaussian noise in 
communication channels, respectively, and G k is the sample of the physical field at the 
location of the K-th sensor. The function G(x, y) is an implicit function of the three unknown 
parameters that have to be estimated at the fusion center. The parameters are ju , the 
strength of the field and (x c ,y c ), location of the object generating the field. 


This problem was formulated and solved using Maximum Likelihood approach, where the 
solution was the set of the three parameters that maximized l(z x ,...,z K ). Since the 
expression for the log-likelihood function was highly nonlinear in unknown parameters, an 
iterative algorithm was used to find the solution. First, a set of Expectation-Maximization 
iterations was formulated and then used nonlinear optimization functions from Matlab 
(nonlinear least squares) to solve for iteratively updated estimates of the parameters. 

4.1.2 Distributed Coding and Estimation 

A numerical solution to the problem of physical field estimation from noisy sparse 
measurements acquired by a sensor network was implemented and demonstrated. This 
estimation problem relied on the following assumption. It was assumed that each object of 
interest was characterized by a physical field. The physical field was measured by a 
heterogeneous wireless sensor network. The density of the network, area covered by the 
network and type of local and channel noise were design parameters and varied. To meet 
the requirement of minimum bandwidth, sensors recorded independent measurements of a 
physical field, then locally quantized them and communicated their quantized 
measurements to the fusion center for joint processing. 

Given noisy quantized measurements of an unknown field (shape was known, but 
parameters were not), the task of the fusion center was to estimate the spatial distribution 
of the measured field. Due to the probabilistic nature of quantized measurements, a 
Maximum Likelihood (ML) estimation approach was adopted. A set of iterative equations 
implementing Expectation Maximization (EM) algorithm was developed. The equations 
were solved numerically by involving Newton’s linearization. 

In the example illustrated below, 15 basic sensors were placed at random within an area of 
size 8 -by -8 meters. The sensor locations were assumed to be known to the fusion center. A 
2D Gaussian distributed field was generated for our experiments and sampled by the sensors 
in the sensor network. Recall that the assumption was that the shape of the field is known, 
but the strength of the field and the location of the object generating the field were unknown. 







To generate the noisy quantized realizations of the sensory data, a realization of Gaussian 
noise was added to every local measurement. Samples were locally quantized by a uniform 
quantizer with 8 levels and transmitted to the fusion center over parallel white Gaussian 
channels with the variance of noise 1. The results of estimation by using the iterative 
approach are shown in Figure 46. With the initial guess of the strength of the field 0 X =12 
and the object location i.d 2 ,0 3 ) = (3,3) it took 450 iterations for the algorithm to converge to 

the final value of the estimates. The true parameters used to simulate the samples of the field 
generated by an object were magnitude 10 and location (4,4). 



Figure 47. Estimated magnitude (left), x-coordinate of the location parameter (middle) and y-coordinate of the location 
parameter (right) displayed as functions of the number of iterations, k. 

The convergence and the convergence rate of EM algorithm depended on its initialization. 
The main benefit of this work is in its potential applications. The estimate of a field could be 
a basis for a variety of inference problems such as detection, classification, tracking and 
object recognition. 

4,1.3 Facial Encoding and Matching Methods 

New face encoding and matching methods operating in near real time were developed and 
tested. Frontal face images were first detected by using a combination of Haar wavelets and 
AdaBoost method, then aligned and rescaled by using software detecting eye location and 
affine transformation. The detected face images were Gabor filtered and then encoded using 
multi-scale Weber operators. The encoded images were subsequently matched against a 
gallery database by involving a symmetric I-divergence measure. Both the scale parameter 
of the operators and their number are tuned to guarantee near real time classification of 
subjects. The operators were applied to encode and match images for both single spectrum 
and cross-spectral (color vs. up to 1650 nm) comparisons. It could also deal with multiple 
scales (different image resolutions). 

New results on the topic of cross spectral face localization and recognition were obtained. 

o Three filter banks: Gabor, Median, and Curvelet were used as initial encoding 
step to encode visible and Short Wave Infrared face images. The secondary 
operators included Weber, Local Binary Patterns (LBP) and Generalized LBP. 
All results were fused at the matching score level. The results indicated that 
(1) without adaptive score normalization (previously developed by our team) 
fusion of the three methods outperform each of the individual filter banks; (2) 























with involvement of the adaptive score normalization, the Gabor filter bank 
outperformed any combination of the three filter banks, 
o An eye localization algorithm applied to visible and SWIR data was developed. 
The detection rate (both eyes were detected) was 70%. 


A set of dimensionality reduction methods was designed to further boost performance of the 
face recognition method. 

Performance of a previously developed iterative method for distributed estimation of a 
Gaussian shaped physical field was evaluated. Both analog and digital cases were considered. 
The performance of the estimator was displayed as a function of the number of quantization 
levels and the number of sensors in the distributed network. The communication links were 
assumed to be parallel Gaussian with known statistics of the noise. 

4.1.4 Multivariate Adaptive Regression Spline Approach 

The problem of composite function (field) estimation in distributed networks was 
formulated as the problem of evaluating the number and locations of the objects generating 
the field. This problem was known as estimation of the model order. In this case it was 
entirely solved by using Multivariate Adaptive Regression Spline (MARS) approach. MARS 
approximated the unknown smooth function in a 2D space as a linear combination of hockey- 
stick functions. It assumed a large number of basis functions supporting the problem of the 
field estimation at first and then pruned uninformative basis functions by using a generalized 
cross-validation method. The results of performance evaluation were available for the case 
of the function approximation problem (no noise is present). 

4.1.5 Fields Generated by Chemical Substances 

The ability of a sparse distributed network of chemical sensors to estimate parameters of a 
field generated by a chemical substance was further analyzed. The work relied on few 
assumptions and models. Sensor responses were modeled as samples of an unknown 
deterministic concentration field generated by a chemical substance placed within the reach 
of the network. Sensor measurements were quantized to save energy and bandwidth during 
transmission of the individual measurements to the fusion center. Transmission channels 
were modeled as parallel Gaussian channels. The fusion center estimated the location and 
the peak value of the unknown field from sparse noisy distributed measurements. The 
solution was due to Expectation Maximization algorithm. The performance of the estimator 
was analyzed as a function of the network SNR, number of sensor nodes and the number of 
quantization levels. 

4.1.6 Stochastic Reduction Algorithm 

A stochastic dimensionality reduction algorithm to improve performance of a previously 
designed algorithm for cross matching visible light and SWIR face images was developed. 
The stochastic model assumed that the light bounced back from a patch on an object was 
(cumulatively, by the law of large numbers) complex Gaussian distributed random variable. 
Pixel intensity, therefore, was the energy of the light reflected from a specific patch on an 



object. A processed SWIR image was compared against a processed visible light image and 
selected segments in both images that were similar in terms of the empirical relative 
entropy. 

The application of new models required design of new optimal processing algorithms. New 
optimal algorithms estimated unknown parametric fields from distributed noisy sensory 
data were developed. 

a. For a field modeled as a Gaussian bell function (a field due to chemical processes), 
an expectation-maximization algorithm was formulated and implemented to 
solve for unknown parameters. The sensitivity of the algorithm to a number of 
parameters, such as signal-to-noise ratios, the number of quantization levels, the 
density of sensors in the network and initial values used by the algorithm to 
estimate unknown parameters, was analyzed. 

b. For a field modeled as an instance of a Poisson counting process (a field due to a 
radioactive object), a compound algorithm was developed that combined the best 
features of a bi-section method and of a secant method. Sensitivity of the 
algorithm to the network parameters was analyzed. 

c. For a magnetic field with the magnitude decaying inversely proportional to the 
third power of distance from the location of the field (a field due to a 
ferromagnetic object), a non-linear regression analysis was applied to estimate 
locations of multiple objects. The dependence of the variance of estimated 
parameters on the number of quantization levels and on the level of noise at the 
estimator end was evaluated. 

4.1.7 Maximum Likelihood Solution 

The convergence of the maximum likelihood (EM) solution to solve the problem of 
distributed target localization was analyzed. This solution was previously proposed by the 
team and applied to estimate Gaussian shaped field from noisy quantized sensory. In the 
best simulations, the iterative algorithm was assigned different initial values for unknown 
parameters. The results of this experiment can be displayed as a function of the MSE vs. the 
radius of the ring within which the initial values are drawn. The two panels in Figure 48 
below show: (left) the distribution of initial values as a function of the ring radius and (right) 
the box plot of square rrror as a function of the radius for the case of 20 sensors and 
SNRo=SNRc=15 dB. In this example, the object is placed at the location (4,4). 





Figure 48.Distribution of initial values as a function of the ring radius the box plot of Square Error as a function of the 

radius for the case of 20 sensors and SNRo=SNRc=15 dB. 

4.2 Subtask 3.2 Middleware services for tracking and recognition applications 
using wireless sensor networks 

4.2.1 Collaborative Multi-view Face Acquisition 

In order to support such real-time face recognition systems, a camera network based data 
acquisition system was designed and implemented that collaboratively acquired multi-view 
face images of a human subject. A choke-point scenario was specifically considered where a 
network of cameras was deployed over small critical access regions such as entrances, 
lobbies, walkways etc. in public places for human identification. The service used the 
geometry of the multi-camera network and inter-camera communication to collaboratively 
acquire both frontal and non-frontal face images in real-time while maintaining a high 
sampling rate. Using this, an image could be processed for detecting non-frontal faces at 
almost the same rate as for frontal faces. At the same time, by narrowing down the potential 
regions in an image for non-frontal face detection, we significantly improved the reliability 
of non-frontal face detection. Our system was easy to setup, did not require camera 
calibration and only depended on fundamental matrices of transformation between camera 
pairs. 

Figure 49 shows the use of camera network geometry for side face detection: When a frontal 
face was detected on any camera, a notification was broadcasted to other cameras specifying 
the center of the detected face. Epipolar geometry was used to project this point to a 
corresponding epipolar line (shown as AB) in the other cameras. The other cameras applied 
a pose-specific side-face detector in a small region surrounding the segment of the epipolar 
line that intersects with the background subtracted image. 





























Figure 49. Epipolar camera comparison. Line AB indicates epipolar line in the other cameras. 


The face acquisition system was implemented on a 1.6 GHz Intel Atom Processor based 
embedded camera network and showed that it can reliably acquire frontal faces at 11 fps 
and non-frontal faces at $10$ fps on images captured at a resolution of 640 x 480 pixels. 

4.2.2 Co-design Framework for Distributed Tracking Using Sensor Networks 

Designing control and tracking applications using a sensor network was a challenging task 
because the target track information needed to be acquired and communicated over multiple 
hops on an unreliable wireless medium prone to collision and fading effects. Therefore, 
information could be error-prone, could have unpredictable delays or even be lost. To 
address this challenge, a co-design approach was adopted in which the application strategy 
for tracking was designed hand-in-hand with the network protocols resulting in guaranteed 
system performance. Such a co-design was needed because a tension existed between the 
application requirements and what the network can supply. 

4.2.3 The Pursuit Application 

An eventual pursuit application was considered as an example. The objective of the pursuit 
control system was for the pursuer object to catch one or more evader objects in the system. 
This system can be seen in Figure 50. The pursuer object was guided towards the evader 
using information provided by a multi-hop wireless sensor network. In this application, if 
the pursuer agents gained perfect information about the entire network instantaneously, 
designing the control strategy became simple. However, imposing such a requirement on the 
wireless network resulted in a lot of contention and therefore ended up decreasing the 
overall system performance. Therefore, the application needed to identify weaker network 
requirements that still resulted in provable convergence properties. These conditions 
imposed a specification for the network layer in terms of network abstractions and these 
abstractions had to be implemented using appropriate middleware services. 
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Figure 50. Exploiting distance-sensitivity for pursuer-evader tracking. 


Regarding the pursuit control application, we showed that eventual catch was satisfied if the 
error in the estimate of distance to the evader decreased linearly with distance between 
pursuer and the evader, if the rate at which this information was supplied to the evader 
decreased linearly with distance and if the staleness in the information supplied decreased 
linearly with distance, where the constants of proportionality depended on the relative 
speeds of the pursuer and the evader. These conditions were represented by a network 
abstraction, namely distance sensitivity. In order to implement these abstractions, we 
designed a middleware service that periodically delivers information while meeting the 
required distance sensitivity properties. 


4.2.4 Exploitation of Distance Sensitivity 

A decomposition technique for solving coupled convex optimization problems over a 
wireless network using local message exchange was developed. To do so, distance sensitivity 
properties inherent in many spatially distributed control systems were exploited, i.e., the 
influence of an actuator on sensors decays with distance and likewise a controller tends to 
be less impacted by sensors that are farther away. The distance sensitivity in actuator 
influence was modeled using an influence matrix whose coefficients decayed away from the 
diagonal at different rates. Distance decaying property was used to decouple the 
optimization problem in such a way that individual actuators decided their control inputs 
based on information exchange within a bounded locality that depended on the nature of the 
influence matrix. In contrast, existing techniques used a network wide exchange of messages 
to achieve convergence, which imposed high control latency. 

4.2.5 Exploiting Distance-sensitivity for Spatially Distributed Control Systems 

For many spatially distributed control systems the influence of an actuator on sensors 
decayed as a non-decreasing function of distance and likewise an actuator tended to be less 
impacted by sensors that were farther away. This property was exploited to identify 
conditions under which local feedback was sufficient for satisfying a global objective. This 
was useful for control systems that operate over wireless networks because communication 
can be restricted to local areas only. The idea was based upon an observation that the 
inverse of a diagonally decaying matrix (with exponential decay) can be approximated by a 
tri-diagonal matrix - which in turn pointed to local feedback being sufficient for maintaining 
optimality in control. 








4,2.6 VCAST Traffic Information Services 

VCAST, an algorithm for obtaining individual vehicle location as well as aggregate traffic 
information over an infrastructure-less multi-hop wireless vehicular network was 
designed. VCAST can be used to improve safety against collisions, inform about 
approaching emergency vehicles and lane merging vehicles (that may even be beyond a 
single hop communication range) and to enable dynamic routing and navigation techniques 
by providing aggregate traffic information in an extended neighborhood. To ensure 
scalability in forwarding information over multiple hops, we exploited a notion of distance 
sensitivity in information propagation, by which traffic information is propagated at a rate 
that decreased linearly with distance from the source. By doing so, traffic information was 
obtained with a staleness, which is a measure of error in the traffic information, that was 
bounded by 0(d 2 ) where d is the communication hop distance from the source of the 
information. At the same time, the required communication rate per unit time at each node 
only depended on the radius of the region in terms of communication hops, and not on the 
vehicular density or the number of vehicles in the region. VCAST did not require any 
special hardware or modification to vehicular transmission standards; instead it could 
simply piggyback on basic Here I am communication for vehicular networks. The results 
were presented at the IEEE Vehicular Technology Conference 2012. Figure 51 depicts 
VCAST’s range and accuracy when used to disseminate traffic information. 
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Figure 51. The figure shows the impact of the communication range on information staleness at different inter-vehicular 
distances when VCAST is used to disseminate traffic information. The source broadcast rate is 5Hz. 

The results which were submitted to IEEE Vehicular Technology Conference 2012 dealt 
with: 

1. Simulations were carried out in ns-3 using an IEEE 802.lip (DSRC) transmission 
model to quantify the impact of channel fading, vehicular density, vehicular mobility 
and time-varying inter-vehicular separations on the achievable staleness. 

2. The average communication cost incurred by each vehicle was quantified. 

3. An analytical comparison of the communication cost with schemes that do not 
incorporate distance-sensitivity was provided. 

4. Complete proofs were developed for all Lemmas and Theorems. The extended results 
were submitted to the international Journal of Vehicular Technology. 

A basic vehicular mobility simulator was developed that allows the team to test reactive 
vehicular systems based on information obtained from VCAST. Note that the concept of 




















VCAST can even be applied in military settings to provide distance-sensitive location 
tracking of personnel and targets. Efforts were also directed at extending the protocol along 
these lines. 


4.2.7 Simultaneous Coverage and Tracking: 

A network of PTZ cameras used for monitoring a large region was considered and a local 
algorithm was described that could track targets at a high resolution while maximizing the 
coverage resolution at all non-target points. The algorithm was local, converged quickly to a 
final state and could remain stable in a configuration for longer periods in the presence of a 
mobile target. The performance of the algorithm was evaluated in simulations, the impact 
of network density was analyzed, and the size of the area over which the reconfiguration was 
performed on the achievable coverage resolution. It was observed that by progressively 
increasing the size of the locality over which the reconfiguration was performed, the 
improvement in coverage resolution started decreasing significantly, thus highlighting the 
merits of a localized approach. As a result, targets could be tracked at a high resolution, the 
entire space could be observed at an acceptable resolution, and the observing cameras could 
be kept stable for long periods of time by only performing the reconfiguration in a local area 
around the target. 
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Figure 52. Impact of the locaclity of reconfiguration on the normalized coverage resolution of a monitored area. 


Figure 52 depicts the impact of the locality of reconfiguration on the normalized coverage 
resolution of an area that is being monitored. As the locality increased, impact on coverage 
resolution progressively started to decrease, highlighting merits of a localized approach. 


4.2.8 Face Recognition 

The use of a wireless camera network was demonstrated to perform near real-time face 
recognition that yields high accuracy and robustness. A portable camera network was 
assembled using off-the-shelf components in which individual cameras perform background 
subtraction and face detection, and transmit only relevant face images to a fusion center for 
recognition. The technique relies on opportunistically acquiring suitable face images as a 
subject walks through the multi-camera network. By only transferring the face images and 
further filtering them based on suitability for recognition, it was possible significantly reduce 
the required network bandwidth. An efficient face recognition algorithm was also designed 





using multi-scale Weber operators. The performance of this system was evaluated for face 
recognition using a dataset of 29 subjects collected using the portable camera network. The 
networked face recognition system is able to achieve comparable recognition performance 
to the commercial LI Facelt software while reducing the required processing time. 

Trident Systems demonstrated their UWB system for acquiring video images in real-time, in 
an outdoor environment. During this demonstration, a few videos were collected with 
human subjects walking through the area at different distances from the camera. The 
objective was to test if the images would successfully work with the multi-view face 
detection system. Figure 53 depicts an individual being photographed from Trident systems. 
We tested those images for front face detection system and the performance was satisfactory 
at distances of approximately up to 10-13 feet. A sample detected face image is shown below. 
The face detection system was developed on a portable Linux platform of about 600 MHz 
and thus potentially the system could be integrated with the Trident UWB system. 



Figure 53. A sample image collected from Trident systems test are. The figure shows the face image and human body 
silhouette detected by our system on a portable Linux platform. 

4.2.9 Control Serializability 

It was identified that ensuring consistency and serializability in the presence of concurrent 
executions was a fundamental problem in a distributed sensor actuator network based 
control system. The camera coverage control system described above was a specific instance 
of such a system. It was shown using a prototype implementation on a network of Telos 
motes that this was expected to significantly decrease the amount of contention and the 
latency of each control action. It was also shown that there are several design parameters 
that impose an inherent tradeoff between the likelihood of inconsistency and achievable 
control latency. The result was to use these parameters to implement a service that can be 
tuned for optimal performance (in terms of control latency) based on deployment density 
and communication range. 

4.2.10 Fusion Techniques for Networked Biometric Systems 

Recent terror attacks, security threats and intrusion attempts have necessitated a transition 
to modern biometric systems that can identify humans in real-time under unconstrained 
environments. Specifically, the following were three critical transitions that were needed 
and which formed the focus of the research: 







1. In contrast to operation in an offline mode using previously acquired photographs 
and videos obtained under controlled environments, it was required that 
identification be performed in a real-time dynamic mode using images that were 
continuously streamed in, each from a potentially different view (front, profile, partial 
profile) and with different quality (pose and resolution). 

2. While different multi-modal fusion techniques were developed to improve system 
accuracy, these techniques mainly focused on combining the face biometrics with 
modalities such as iris and fingerprints that were more reliable but require user 
cooperation for acquisition. In contrast, the challenge in a real-time networked 
biometric system was that of combining opportunistically captured multi-view facial 
images along with soft biometric traits such as height, gait, attire and color that did 
not require user cooperation. 

3. Typical operation was expected to be in an open-set mode where the number of 
subjects that enrolled in the system was much smaller than the number of probe 
subjects; yet the system was required to generate high accuracy. 

To address these challenges and to make a successful transition to real-time human 
identification systems, following contributions were made: 

1. A score-based multi-modal, multi-sample fusion technique was designed to combine 
face images acquired by a multi-camera network and the effectiveness of 
opportunistically acquired multi-view face images using a camera network in 
improving the identification performance is characterized; 

2. The multi-view face acquisition system was complemented by a network of Microsoft 
Kinects for extracting human anthropometric features (specifically height, shoulder 
width and arm length). The score-fusion technique was augmented to utilize human 
anthropometric data and the effectiveness of this data was characterized. 

3. The performance of the system was demonstrated using a database of 51 subjects 
collected using the networked biometric data acquisition system. 

The results showed improved recognition accuracy when face information from multiple 
views was utilized for recognition and also indicated that a given level of accuracy could be 
attained with fewer probe images (less time) when compared with a uni-modal biometric 
system. Furthermore, it was observed that when the required false acceptance rate was very 
low (<1%), human anthropometric data plays a critical role. 

4.2.11 Demonstration of Real-time Face Recognition at DHS Center of Excellence 
Meeting 

The previously designed collaborative multi-view face acquisition system was integrated 
with PittPatt face recognition software, and the combined system was used to perform real¬ 
time face recognition in an unconstrained setting. The combined prototype system was 
demonstrated at the DHS center of excellence performance review meeting held at Phoenix, 
AZ on August 15,2011. Only the front face images were used for recognition. The system was 
tested with a suspect database of 3 subjects and showed high accuracy (<5% False positives 
and almost 0% False rejection rate). 



4.2.12 EzTrack: Indoor Positioning 

Radio frequency (RF) based techniques, being low cost and not requiring line of sight, were 
ideally suited for indoor tracking of humans and objects in various enterprises such as 
hospitals, office spaces, assisted living centers and child care centers. However, RF based 
tracking was also challenging because radio signals are prone to time-varying multipath 
interference in indoor environments. To counter this challenge, most existing techniques for 
RF based indoor tracking relied on building a RF signature model of the indoor environment 
using offline calibration, which can be a time consuming and expensive process and still 
cannot be exhaustive. Requirements of prior site survey and calibration severely limited the 
portability of RF-based solutions and have inhibited the widespread deployments of RF- 
based indoor tracking systems. Here, EzTrack was presented; an adaptive and self¬ 
configuring RF-based tracking system, which utilized wireless sensor networks, which 
eliminated the need for prior calibration. EzTrack was able to achieve 75 th percentile 
accuracies of about 4.5-7.5 feet (1.5-2.5m) in indoor environments. The performance of 
EzTrack was validated using experiments in different indoor environments with networks 
of IEEE 802.15.4 based TelosB motes. Figure 54 depicts error tracking. 



Figure 54. Tracking error in EzTrack - anchor nodes at 14 feet separation. 


4.3 Subtask 3.3 Decentralized Control Algorithm for Seif-Healing/Self Forming 
Network of Mobile Agents 

A mobile wireless sensor network (MWSN) consists of n mobile sensors (or agents), which 
are connected through wireless communication links along which information flows. These 
sensors act cooperatively to provide distributed, real-time information from the physical 
world. Mobile robots or vehicles are exploited so that the sensors can dynamically reposition 
themselves to maximize the collective information gained by the network. A wide spectrum 
of applications of this structure of MWSN have been reported in the articles which include 
target tracking, formation and coverage control, environmental monitoring and several 
others. For these cooperative sensing applications, it was desirable to maintain a connected 
communication graph, even as communication links were established or lost as the agents 
moved. For this reason, consensus algorithms were used to maintain the connectivity of the 

































































graph. Moreover, communications within the WSNs suffered from unpredictable delay, loss, 
and energy constraints. These constraints also included actuators and energy [battery, fuel) 
limitations, which produced forces, which led to agent movements. So the controller which 
was applied to maintain the connectivity graph should technically consider these constraints 
and uncertainties too. 

4.3.1 Distributed Model Predictive Control 

Many control laws were proposed for controlling a network of dynamic agents with different 
models and communication topologies. Most of them did not exploit optimal control ideas 
and did not account for input constraints, which in many cases had to be included in the 
problem formulation due to actuator limitations and energy efficiency concerns. Distributed 
model predictive control (DPMC) algorithms for consensus problems of mobile agents 
considering delay and uncertainty resulting from wireless communication networks along 
with minimization and equalization of energy consumption of each agent in presence of 
practical constraints on actuators was proposed. 

In multi-agent MPC, each MPC was associated with a different node and computed the local 
control inputs based only on the states of the node and of its neighbors. In this way, actions 
which an agent takes were influenced by both the evolution of the agent itself and the 
evolution of the neighbor agents connected to it through communication links. Since the 
agents in a multi-agent control structure usually have no global overview and can only access 
a relatively small number of sensors and actuators, predicting the evolution of a sub network 
over a horizon involved even more uncertainty than when a single agent was employed. The 
control structure for this system can be seen below in Figure 55. 


control ytiucture 



Figure 55. Distributed multi agent Model Predictive Control Structure. 

The multi-agent model predictive control scheme was defined to achieve the fire tracking 
mission where each robot needed to know its current states, its neighbors’ current states 
and models and constraints of its neighbors. Based on such information, each robot 
computed its optimal inputs and its neighbors’ optimal inputs assuming a constant set of 
neighbors over the horizon. The input to the neighbors was only used to predict their 
trajectories and then discarded, while the first component of the optimal input of problem 


















































was implemented on each robot. The solution of each sub-problem yielded a control policy 
for each robot. The objective was defined as: 

1. Tracking control and area coverage- the main goal of cooperative robot agents was 
to scan an area to collect data. For this reason, they should have track a pre¬ 
specified path which was calculated by path planning algorithms. The algorithm 
worked based on the data gathered from sensors and the planned path was defined 
so that maximum data was gathered from the environment. 

2. Energy minimization- energy minimization was met by minimizing the work that 
was the amount of consumed energy by each agent during prediction horizon. 

3. Equal velocity and graph connectivity- mobile agents should have the same velocity 
so that the overall flocking has monolithic movement leading to mobile agents did 
not deviate and become isolated from the connected graph. 

4. Collision Avoidance- collision avoidance between robots during maneuvers was 
defined as a cubic protection zone for each of the robots so that these zones did not 
interfere during maneuvers as seen in Figure 56. Infinite norm was used to describe 
the collision avoidance constraint since it can be converted to Mixed Integer Linear 
Programming [MILP] form for which several algorithms exist, as opposed to 
Euclidian norm [2-norm] that led to nonlinear non-convex optimization problems 
which were difficult to solve. 



Figure 56. Protection zone of the robots and their position. 

A fire monitoring mission using MWSN utilizing thermal or visual sensors to detect the fire 
was considered as the simulation scenario. Location of the fire was determined by flocking 
of the mobile robots in an area. On the detection of fire by agents, the fire-detection message 
containing the relative direction of the fire location was communicated through wireless 
links to the neighboring agents. Assuming that MWSN formed a connected-graph, the 
message was received by all mobile agents and hence the direction of flocking of the robots 
changed to move to the fire location. Figure 57 [left] a shows the evolution of position of two 
mobile agents in time with corresponding control signal. As seen from the left portion of 
Figure 56, the two mobile agents maneuvered to meet the collision avoidance while fulfilling 
the mission. It noted from the snapshots of the maneuver in x-y plane in Figure 56 [right] 
that the two robots did not collide into each other during their maneuvering. 
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Figure 57. Left, evolution of the robots position in xy plane over a period of time with collision avoidance. Right, collision 
avoidance between two mobile robots; robot 1 (blue) and robot 2 (red). 
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Figure 58. Uncertainty condition in one time stamp: Left, evolution of the robots position in xy plane over a period of time 
with collision avoidance. Right, collision avoidance between two mobile robots; robot 1 (blue) and robot 2 (red). 


Figure 58 depicts the uncertainty condition of the robots in a onetime stamp scenario The 
left portion of this figure shows how the robots positioned themselves in the XY plane over 
a period of time without coming into contact with one another. The right portion of Figure 

58 highlights the areas of collision avoidance between the two robots. Alternatively, Figure 

59 depicts the uncertainty condition of the robot in a multiple time stamp scenario. 
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Figure 59. Uncertainty condition in multiple time stamp: a) Left, evolution of the robots position in xy plane over a perdiod of 
time with collision avoidance. Right, collision avoidance between two mobile robots; robot 1 (blue) and robot 2 (red). 


4.3.2 Autonomous Robots 

Autonomous robots are robots that can perform desired tasks in unstructured environments 
without continuous human guidance. In recent times, autonomous robots have been used 
everywhere, but their main usage is for military purposes like information gathering, 
mapping, and even as weapons. Information gathering is a task done by robots and 
autonomous vehicles to build up a map within an unknown environment, or to update a map 
within a known environment, while at the same time keeping track of their current location. 
This is important in military applications as important data can be collected without risking 
human lives. Our research focused on the formulation of a strategy for the trajectory 
planning problems for autonomous robots in the information tasks where the positions of 
information points are uncertain. This automatically led to model predictive control strategy, 
which used the updated knowledge about the model to solve a finite horizon optimal control 
problem at each time step and only executed the first control strategy. Then this strategy 
was applied to an autonomous robot, collecting information within a finite region with 
randomly changing information points and their main objective function was to collect the 
maximum information in the minimum amount of time. 

4.3.3 Decentralized Model Prediction Control 

Multi-Parametric toolbox was explored to realize the Decentralized Model Prediction 
Control (DMPC). Multi-Parametric toolbox was developed by the Swiss Federal Institute of 
Technology and allowed for the derivation of real-time controller parameters. Tractable 
state feedback controller was obtained with different schemes of infinite time optimal 
control, finite time optimal control, minimum time control and low complexity controller. 
Finite time optimal control was modeled and implemented for tracking control. 

The global model for tracking control was decentralized to obtain a decomposition model of 
two mobile agents. These agents, aside from interacting with the external environment, 
interacted with one another based on predictive control for the tracking mission. Note that 
the decentralized models began to overlap as they shared common states and inputs. The 
optimal input vector was calculated from the predictive horizons for future time instants and 
for each agent only the first component was actually applied. The different simulation 
outcomes were investigated for two control actions, which were: 

1. fast and slow trajectory information sharing, and 

2. fast traj ectory information sharing. 







































































































































































































Fast and slow trajectory information sharing allowed predictive control for slow mobile 
agents with future time instant information from fast mobile agents. The performance index 
increased as slow mobile agent trajectories were controlled utilizing future and current 
information. Figure 60 shows the proposed control where the red indicated faster predictive 
information gathering and blue indicated control based on local and predictive information. 
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Figure 60. Demonstration of Faster and Slower Movement; robot 1 (blue) and robot 2 (red). 


DMPC was based on iterative, finite horizon optimization of a plant model. At time, t, the 
current mobile agent was sampled and a cost minimizing control strategy was computed (via 
a numerical minimization algorithm) for a relatively short time horizon in the future: [t, t+T]. 


In DMPC, only the first step of the control strategy was implemented, then the plant state 
was sampled again and the calculations were repeated starting from the new state, yielding 
a new control and new predicted state path. The prediction horizon shifted forward and for 
this reason DMPC was also called receding horizon control. 

Distributed Model Predictive Control can best summarized as a multivariable control 
algorithm that uses: 

o an internal dynamic model of the process 
o a history of past control moves and 

o an optimization cost function J over the receding prediction horizon 
And, DMPC’s optimum control moves possessed: 

o DMPC methods to reduce the complexity of the mathematical model, 
o DMPC methods could be easily implemented for real time applications, 
o DMPC allowed the agents to make decisions onboard without the involvement 
of the operator. 


































































































































4.3.4 Summation of the Functioning of MPC on Any Plant/System 

MPC is an established control technique because of its prominent advantages in solving 
problems with physical constraints as well as multivariable problems in nature. Subject to a 
set of constraints, the objective of MPC is to minimize an appropriate cost function in an 
online fashion by determining the optimal input sequence over the control horizon. The cost 
function usually consists of two components: (i) one is the tracking error between the 
predicted process output and the reference trajectory (also known as set points) over the 
prediction horizon; (ii) the other component is the incremental control moves over the 
control horizon. 

4.3.5 Extended Kalman Filtering SLAM using MPC with Dynamic Features 

Here, the case considered was when the information gathering task was formulated by 
estimating a state vector containing features of interest in the environment. The estimation 
was performed by Extended Kalman Filter (EKF). It was showed that the planning problem 
in this scenario could be regarded as an optimal control problem for a nonlinear system with 
a gradually identified model. Then, the MPC strategy was proposed, which was suitable for 
online decision making, changing models and dynamic constraints. Two scenarios, SLAM 
with single robot and with two robots were used as case studies. 

In order to implement the Kalman Filter in the SLAM process, it was necessary to describe 
the system. Therefore, it was assumed that the robot moved in a linear fashion, and that the 
system could be described by the following equations: 

Equation 3 

Xk+l = Fk x k + Gkuk + Vk 


Equation 4 

yk = HkXk + Wk 

Equation 3 was the state equation and indicated the robot state at the next time step is a 
linear function of the current state, Xk, the input vector, Uk, and an error term, vk. Equation 4 
was the output equation and indicated that the measurement vector taken from the sensors 
was a linear function of the robot state, Xk, and an error term, Wk [17,18]. 

The constraints of the robot’s motion were to be incorporated into the planning process. The 
control constraints and state constraints can be expressed by 


Equation 5 

uk e Uk, x(t) e X(k+i), vt e [k,k+l], 

where Uk is the set of admissible controls for robot at time k and X(k+i) describes the safe 
region for robot during time k to time k+1. 
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robot and landmark location is required. 


Figure 61 below is the simulation results for the SLAM using Kalman Filter for a single robot, 
the first figure on left shows the trajectory followed by a robot having limited sensor range 
and the second figure on right shows the trajectory followed by a single robot having wide 
sensor range. 
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Figure 61. SLAM with EKF and without MPC (left) and b) SLAM with EKF and MPC (right) 

4.3.6 Kalman Filter Derivation 

The robot was never aware of its state with absolute certainty due to measurement and 
process errors. Therefore, the robot state was tracked as the mean of a normal distribution. 
Now, since the robot state was a vector containing multiple variables, the mean of the joint 
distribution of these variables had to be tracked [19]. 

The Kalman Filter worked by predicting the robot state at the next time step based on known 
information, and then corrected this prediction at the following time step based upon 
measured data at that time. Therefore, we had to find an equation for the prediction of the 
robot state. 



The robot state was predicted by plugging the current robot state into the state equation. We 















also know that the process noise has mean 0, so we predicted vk to be 0 and found: 


x~k+l = Fkx'k + Gkuk 


Uk Uk-1 Uk-2 


Xk: Xk f Xk-i Xk-2 y 
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Figure. A graphical model of EKF SLAM algorithm. 

Figure 62. Graphical model of EKF SLAM algorithim. 


Note here that the hat on x denoted that it was 
an estimate rather than the actual state, and 
that the subscript denoted that it was an 
estimate of the state at time step k + 1. It 
should also be noted that we used superscript 

- (e.g. x" - k ) to denote that the estimate was 
performed with information up to time k - 1. 
This made both the notation x" - k and x\+i 

predictions since they were estimates of a 
certain time step with information up to the 
previous time step. 


4.3.7 Research Issues on Mobile Wireless Sensor Networks 

Two main research issues were communication issues and data management issues. 
Communication issues included topology control, coverage, target tracking as well as 
localization. Data management issues were data gathering and data replication [18]. 
Topology control was the problem of assigning transmission powers to every node in order 
to maintain connectivity while minimizing the energy consumption of the whole network. 

• One of the basic and significant factors in the design and application of MWSNs was 
sensor coverage measured by the overall area that a WSN currently monitored. 
Sensor coverage was closely related to the quality of service that the network could 
provide, and it decreased due to undesirable sensor deployment and sensor failures. 
There were two ways to maintain sensor coverage in these cases: self-deployment 
and relocation. 

• Self-deployment meant that mobile sensors should have the ability to autonomously 
adjust their positions to improve the coverage after their initial deployment. 

• Sensor relocation refers to deploying a moderate number of redundant sensors and 
strategically relocating them as needed to fill the position of failed nodes. 

• Target tracking was the need for sensors to be aware of the position of nodes in order 
to make sense of data and perform further navigational tasks. Because of mobility 
which increased the uncertainty of nodes, localization in MWSNs was more difficult. 






4.3.8 Data Replication Issue 

In MWSNs, due to the free moving of sensors and harsh environments, disconnection 
occurred frequently which resulted in network divisions. In such situations, sensors in one 
of the divided network could not access data in other divided networks. One to solve the 
network division problem for improving data accessibility was to replicate data items from 
other sensors. 

4.3.9 Model Predictive Control of a SISO 

The main objective was to hold a single output, y(bar), at a reference value (or set-point ), r, 
by adjusting a single manipulated variable (or actuator) u. The SISO plant actually had 
multiple inputs, as shown in the following figure. In addition to the manipulated variable 
input, u, there was a possible measured disturbance, v, and an unmeasured disturbance, d. 
Model Predictive Control Toolbox design always provided feedback compensation for 
unmeasured disturbances and feed-forward compensation for any measured disturbance. 
Model Predictive Control required a model of the impact that v and u have on. It used this 
plant model to calculate the u adjustments needed to keep y(bar) at its set-point. This 
calculation considered the effect of any known constraints on the adjustments (typically an 
actuator upper or lower bound, or a constraint on how rapidly u can vary). One may also 
specify bounds on y(bar). These constraint specifications were a distinguishing feature of 
Model Predictive Control. 


Measured Disturbance 



Measured Output (Controlled Variable) 


Figure 63. Simple SISO system model with plant and MPC control. 

Typical Sampling Instant-Model Predictive Control Toolbox design generated a discrete-time 
controller—one that acquired action at regularly spaced, discrete time instants. The 
sampling instants were the times at which the controller acted. The interval separating 
successive sampling instants was the sampling period, A t (also called the control interval). 

Figure 64 shows the state of a SISO model predictive control system. This system operated 
for many sampling instants. Integer k represented the current instant. The latest measured 
output, yk, and previous measurements, yk-i,yk- 2 ,..., are known and are the filled circles in the 
figure. Figure 64 also shows the controller's previous moves, uk-41,..., uk-1, as filled circles. 
To calculate its next move, uk the controller operated in two phases: 

o Estimation. In order to make an intelligent move, the controller needed to know 
the current state. To accomplish this, the controller used all past and current 
measurements and the models. 

o Optimization. Values of set-points, measured disturbances, and constraints were 
specified over a finite horizon of future sampling instants, k+1, k+2,..., k+P, where 
P (a finite integer > 1) is the prediction. The controller computes M moves uk, uk+1, 















... uk+M-1, where M ( > 1, < P) is the control horizon. In the above example shown 
in the figure, P = 9 and M = 4. The moves are the solution of a constrained 
optimization problem. 

When it finished calculating, the controller sent move Uk to the plant. The plant operated with 
this constant input until the next sampling instant, At time units later. The controller then 
obtained new measurements and calculates new control input. This cycle repeats indefinitely. 

Controller State at the kth Sampling Instant 



-4 -3 -2 -1 k +1 -i-2 +3 +4 +.t> +6 +7 +8 +9 

Sampling Instants 

Figure 64. Illustration of MPC. 



Figure 65. Servo responses. 


The servo response shown in Figure 65 settles within 5 seconds with no overshoot. It also 
displays the torque's lower bound, which became active after about 0.9 seconds but wasn’t 
exceeded. The applied voltage saturates between about 0.5 and 2.8 seconds, but the 
controller performs well despite this. 

4.3.10 Multi-rate Fast Output Feedback Sampling Controller 

In a discrete time system with input Uk, states xk and output yk at time t = kx. 

Xk+i = OxXk + r T Uk 
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Assuming that a state feedback gain F has been designed such that (O x + r T F) has no eigen¬ 
values at the origin. For this state feedback, one can define the fictitious measurement matrix 
C(F, N) = (Co + DoF) (<D t + r T F) _1 , which satisfies the fictitious measurement equation yk = C 
xk then the static output feedback for the system (1) will be Uk = L yk. For L to realize the 
effect of F, it must satisfy 

Xk+i = (Ox + TtF) Xk = (Ox + TxLC) Xk i.e. LC = F. 

Let Y denote the observability index of ((|), T). It can be shown that for N > y, generically C 
has full column rank, so that any state feedback gain can be realized by a fast output sampling 
gain L. 

In particular, there are many systems in which faster output sampling periods cannot be 
taken, even though the input actuating period can be taken at relatively high speeds. 
Generally in such systems, feedback controllers are designed according to a single rate based 
on the measured output at slow sampling rate. In these cases, the control performance within 
the slow sampling period was not always ensured. However, if the input can be updated at a 
faster rate, one can expect an improvement in the control performance. With this in mind, 
multi-rate control approaches with a lifting technique in which the control input can be up 
dated at fast rate have widely researched. By using the lifting technique, one can design a 
controller with the fast-rate updating rate. 



5 Task 4: Design and demonstrate massively parallel simulation of 
large-scale sensor networks 

5.1 Ad-hoc Network 

In consultation with ARL fellow Dr. Don Torrieri, a very general system model was developed 
for the frequency-hopping ad-hoc network. The channel was designed to undergo Nakagami 
or Rayleigh fading. The effects of adjacent channel interference (due to the spectral splatter 
effect) were taken into account. The modulation and coding were taken into account by 
using the concept of modulation-constrained capacity, which predicted the performance of a 
capacity-approaching code combined with an actual modulation. Given a fixed network 
geometry and set of shadowing coefficients, the outage probability was found in closed form 
for Rayleigh fading, and more generally, Nakagami fading. 

An example is shown in Figure 65. The network consists of 10 interferers located on a disk 
of radius 30 and a source transmitter at unit distance from a receiver at the center of the disk 
(distance is normalized so that one "unit” of distance corresponds to the source-receiver 
distance). The particular network geometry was realized by independently drawing each 
node's location from a uniform distribution. Once the network was drawn, it was fixed for 
the remainder of the analysis. The network geometry for this example was shown in the 
inset of the figure. In the example, the path loss coefficient was equal to 3 and the number 
hopping frequencies was equal to 1000. An outage was declared whenever the instantaneous 
signal-to-interference and noise ratio (SINR) dropped below the threshold p. The outage 
probability was denoted by the variable s. The figure showed the outage probability in 
unshadowed Rayleigh fading for three values of SINR threshold p as a function of the SNR T. 
For each value of p, an analytical curve is shown, which was determined by using the exact 
expression derived during this phase of the research. Also shown is a curve generated by 
simulation, which involved randomly generating the exponentially-distributed power gains 
associated with the channel between each transmitter and the receiver. As can be seen in 
Figure 66, the analytical and simulation results coincide, which was expected because the 
analysis was exact. Any discrepancy between the curves is attributed to the finite number 
of Monte Carlo trials (one million trials were executed per SNR point). 



Figure 66. Outage probability 0 as a function of SNR for SINR thresholds 0= {-10, 0,10} dB. Analytical curves are solid, 
while dots represent simulated values. The network geometry is shown in the inset. 



Figure 66, above, depicts outage probability s as a function of SNR for SINR thresholds (3= {- 
10, 0, 10} dB. Analytical curves are solid, while dots represent simulated values. The 
network geometry is shown in the inset. The star at the center of the radius-30 circle 
represents the receiver, while the ten interferers are shown as dots. The desired transmitter 
is not shown, but lies at unit distance away from the receiver. 

A similar analysis was developed for the more general case of Nakagami fading and log¬ 
normal shadowing. The general outage probability was a function of the particular network 
geometry and set of shadowing coefficients. In addition, the average outage probability was 
derived for unshadowed Rayleigh fading when the nodes are distributed according to an 
independent and uniform spatial distribution. 

The approach for modeling and analyzing ad hoc networks that were discussed above have 
been generalized to account for a variety of channel models, physical-layer waveforms, and 
MAC-layer protocols. The basic approach was to place a reference receiver at the center of 
the network, and then compute the probability of outage at that receiver, where an outage 
was defined to be the event that the signal-to-interference and noise ratio (SINR) falls below 
a threshold value. The threshold value was determined from information theory and 
represented that capacity of the underlying modulation format. The analytical challenge was 
to derive closed-form expressions for the cumulative distribution function (CDF) of a 
random variable which represented the difference between the received signal power and 
the total interference power. By rigorously applying classical probability theory, a closed 
form expression was derived which could determine the outage probability conditioned on 
a particular network topology. 

5.1.1 Guard Zones 

The analytical approach enabled the influence of system-wide parameter settings to be 
investigated. Performance was computed for a variety of code rates, modulation parameters, 
spread-spectrum settings, and MAC protocols. Particular emphasis was placed on 
determining the impact of the guard zone on system performance. A guard zone is a circular 
area placed around a transceiver over which no interferers may transmit. Guard zones were 
implemented in practice by using collision avoidance protocols such as carrier-sense 
multiple access with collision avoidance (CSMA/CA). Figure 67 shows an ad hoc network 
with guard zones. The reference receiver is represented by a star, while transmitters are 
represented by circles. The desired transmitter is red, while interferers are blue. Because the 
nodes represented by green circles are inside the guard zones of the blue interferers or red 
transmitter, they are not allowed to transmit and therefore do not produce interference 
(note that in the absence of a guard zone, the green nodes would transmit and create 
interference). 




Figure 67. An example ad hoc network. 


An example ad hoc network is shown in Figure 67. The reference receiver is the red star, 
and the corresponding reference transmitter is the red circle to its right. Blue circles 
represent interferers. Each transmitter has a circular guard zone surrounding it. Green 
circles are nodes whose transmissions are suppressed due to being in a guard zone. 

A natural question we asked when using guard zones was "what was the optimal guard zone 
radius?” Answer, "there was a tradeoff.” If the guard zone radius was too small, then there 
were nearby interferers that degraded performance, yet if the guard zone was too large then 
not very many nodes were permitted to transmit, reducing the total amount of data 
supported by the network. Using our analytical approach, the optimal guard zone radius 
could be found. 

5.1.2 The Analytical Approach 

First, a network topology was drawn at random according to a representative spatial model. 
The outage probability was determined for the drawn network. Initially, no guard zones 
were used, and the corresponding outage was found. Then nodes were deactivated one at a 
time by imposing guard zones of increasingly larger radii. The corresponding outage of the 
thinned network was computed, and the number of nodes and guard zone radius were noted. 

Outage alone was not a good measure of performance. This was due to the fact that outage 
was minimized by using a guard zone of infinite radius, which suppresses all interferers. 
However, an infinite guard zone was not efficient because only a single transmission was 
supported by the network at a time, which was a poor use of spectrum. Instead of using 
outage, the transmission capacity as a measure of performance was used. The transmission 
capacity was the throughput of the single reference link multiplied by the density of the 
network, measured in the number of nodes per unit area. Transmission capacity was 
interpreted as the area spectral efficiency of the network and takes on units of bits-per- 
second per Hertz per square-meter, or bps/Hz/m 2 . 

Figure 68 shows the outage probability and transmission capacity of a representative 
sampling of networks. Both metrics are shown as a function of the guard zone radius r m in and 


the number of active nodes M. From the figure, it can be seen that setting r m in = 1.2 maximizes 
the transmission capacity, and resulted in a network with 22 active nodes. 




min 



Figure 68. Influence of guard zone radius r m m on network performance. 


Influence of guard zone radius r m in on network performance. The network initially has M=100 
interferers and is contained in a circular region of radius 4. A guard zone of radius rmin is 
imposed. The value of rmin is gradually increased, and the corresponding number of active 
nodes M is noted, along with the outage probability s and transmission capacity x. From 
these curves, the optimal value of r m m =1.2, for which M=22 nodes are active. 


5.1.3 Guard Zones in Sensor Networks 

Interference is a major limiting factor in any ad hoc or sensor network, and the goal of 
medium-access control protocols (MACs) was to control interference. New methodology 
was applied for analyzing ad hoc networks to study critical aspects of interference-avoidance 
protocols. This work was done jointly with ARL fellow Dr. Don Torrieri, and was submitted 
to Infocom-2012. 


In this example provided by Figure 69, a reference receiver was located at the center of the 
network and a transmitter was placed one unit away (to the Thirty (30) interfering 
transmitters are placed in the arena surrounding the reference receiver. These interferers 
were placed at random, but in such a way that they were never too close together. The 
minimum distance between interferers was called an exclusion zone, and is meant to model 
the natural spacing that occurs when wireless networks were deployed. 













Fig. 1. Example network realization. Af = 30 interferers are placed 

according to the uniform clustering model on a circle with radius r-n e t = 6. 
Each mobile has an exclusion zone (not shown) of radius = 0.5. 

The reference receiver (indicated by die yz) 1S placed at the origin and 
the corresponding reference transmitter is one unit away (indicated by the 
• to the right of die receiver). Active transmitters are filled circles, while 
deactivated transmitters are unfilled. A guard zone of radius r g = 1.5 
surrounds each active transmitter, as depicted by the dashed circles around 
the active transmitters. Transmitters that fall in the guard zone of an active 
transmitter are deactivated. 


Figure 69. Example of network realization. 


5.1.4 CSMA Guard Zone 

In addition to the exclusion zone, another zone can be seen in the figure called a CSMA guard 
zone. The CSMA guard zone was a circular area surrounding active transmitters. Any other 
transmitter inside an active transmitter’s CDMA guard zone was not permitted to transmit. 
In the figure, there are 15 active interfering transmitters (represented by filled black circles) 
and 15 transmitters whose transmissions are suppressed because they are in the CSMA 
guard zone of an active transmitter. Figure 69 represented a snapshot in time. During a 
different time slot, a different set of nodes may have become active, while others may have 
become suppressed that are in the new active transmitters’ guard zones. 

In considering such a network, the radii of the two guard zones was critical. For instance, if 
the CSMA guard zone was too large then too many transmissions will be suppressed, but if 
the guard zone was too large, the interference level may be too high. Using the previously 
developed analytical approach and our cluster-computing resources, we determined optimal 
guard zone radii under a wide range of channel and network conditions. The optimization 
was performed with respect to the transmission capacity, which is the area spectral efficiency 
in units of bits per second per Hertz per square-meter (bps/Hz/m 2 ). 



5.1.5 Improvement of Cyber infrastructure 

A significant effort was directed towards testing and improving the high-performance 
cyber infrastructure in place to support the optimization and simulation jobs required to 
enable the team’s research progress. Previously, the cyberinfrastructure was configured to 
execute specific functions related to network optimization, a refactoring of the software 
now allows the computing infrastructure to be able to handle a much wider range of 
computing tasks. As a case study to demonstrate the flexibility of the infrastructure, the 
computing cluster was configured to support the semester course project in the Spring 
semester 2012 Coding Theory course (EE 567). Students would upload the LDPC-code 
parity-check matrix that they designed to a web app, and the code would be simulated in 
parallel on the cluster. Jobs completed in about 15 minutes, in contrast to the 7-8 hours 
required to run locally. CML Matlab Toolbox was utilized. CML is a Matlab toolbox 
designed by the Wireless Communications Research Laboratory (WCRL) for supporting 
various link-level simulations. Initially, the program could only be run on a single 
computer, but WVU now fully runs it parallel on the cluster. 

Capabilities for running parallel Monte Carlo simulations and numerically-intensive 
analytical jobs on the cluster computer were also improved. Focus was on efficient 
scheduling algorithms. The approach to scheduling is documented in the thesis of Aruna 
Bommagani, a M.S. student funded by this project who successfully defended. The previous 
capability to run parallel Monte Carlo simulations on a cluster computer was also 
generalized, so that they could now run on a very general class of scientific computing 
applications written in Matlab on our cluster. 



6 Conclusion 

Four tasks were addressed in the course of the Advanced Communication for Wireless 
Sensor Networks projects supported by the Army Research Laboratory. These tasks 
included: 

Task 1: Design, test and demonstrate cooperative and multi-antenna communications 
systems. 

Task 2: Design, test, and demonstrate low-power analog and compressive sensing 
subsystems. 

Task 3: Design, test and demonstrate real-time fault-tolerant protocols for data 
aggregation, inference, location tracking, and connectivity maintenance in a mobile 
heterogeneous sensor network. 

Task 4: Design and demonstrate massively parralell simulation of elarge-escale sensor 
networks. 

To address Task 1, two subtasks were undertaken, rapid prototyping of MIMO systems and 
investigation of cooperative communication systems. Task 2 was addressed with the 
assistance of three subtasks, nano-derived micro sensors for remote threat detection, 
reliable and energy efficient sensing using analog signal processing, and low-power 
collective sensing. Research focused on task 3 was comprised of three subtasks, 
distributed coding and data exchange in heterogeneous sensor networks, distributed target 
tracking and connectivity maintenance in a mobile network, decentralized control 
algorithm for self healing/self forming network of mobile agents. Finally, task 4 did not 
have any defined subtasks, however focus was placed on ad hoc networks, guard zones, 
and the improvement of cyber infrastructure. 
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